[Freeipa-devel] Paging in Web UI

[Petr Vobornik]

I would like to point out a problem in Web UI related to paging and 
suggest a possible solution:

Current implementation of paging in Web UI is not serving it purpose 
which should be to do operations faster and be able to find all users.

How current implementation should work:
  1) get all primary keys (pkeys) of certain object type (users, ...)
  2) load first 20 of them
  3) from 1) we know record count so we can choose other page (different 
subset of 20)

How it actually works:
  * get up to 2000 keys, with more items a LDAP limit is reached and 
result is truncated
  * we don't know how many are there, paging doesn't work as it should

Problem:
Let's say that paging is working (no LDAP limit) and consider this: Our 
page size is 20 records. With 400 users it is 20 pages. With 2000 users 
(current search limit) it is 200 pages and so on. Personally I wouldn't 
like to go through 20 pages. 200 or more pages sounds like a punishment. 
There is also a question if we even need to do it. I don't see a benefit 
of paging here. For smaller setup an increase of page size may help 
admin to see more users on a page more quickly (rolling mouse wheel is 
faster than click on next page button).

For large setups, size of the pkey list must also be taken on mind. For 
each record there is a pkey and dn (dn is not required but it would need 
some hacking in python code to remove it). At the moment the list size 
raises drastically because JSON response includes indenting by spaces 
and is not compressed (easy to fix, but still there). With 10 000 or 
more users this pkey and dn list is pretty big (slow load on slower 
networks -vpns). This list is also loaded on each page change 
(record-wise, can be improved - cached).

IMO with hundreds or thousands of users the most common use case is 
search by login or name or something. Paging is not required this case. 
It may be required if the result count is greater than size limit. In 
such case an option to temporary increase size limit or enable paging 
would be beneficial. Apart from this case, paging should be off. It will 
speed up page load because it executes only one request.

Possible solution:
1) I suggest to introduce configuration options for paging. They can be 
global (default for all search pages) or individual for pages or Web 
UI's users. Per-user configuration can be stored in browser (to not 
pollute LDAP with application stuff). Global configuration on server in 
config plugin (minor polluting).

Those options should be (per page and global):
  * enable paging
  * page size

Note: when paging is disabled page size is basically --sizelimit option.

2) On search pages we should have controls to enable/disable paging and 
to change sizelimit for this particular moment.
3) Compress JSON responses (on httpd level)

This way admin can configure default's for his deployment and user's can 
adjust for certain situations.

Btw always including member_xx attributes in find or show commands is 
not good for search pages either but that's another topic.

Comments are welcome.

Regards
-- 
Petr Vobornik