REST API / Project list only paged

[ddmesh]

Hi,
when I use the REST API to retrieve ALL projects via json, I only get the first few.
I do not specifiy any page parameter in URL. So I expect a json containing all projects.

Our environment only creates some projects (around 30) and I have to do three tries to retrieve
all projects. But in meantime when requesting page 2 other projects might be added/deleted.
This would cause an invalid "snapshot-view" of all projects.
Can you change the REST API in this way that without any page-parameter all projects are included in returned json object?
If I need it paged than I would specify the page parameter and perhaps a parameter to tell how many entries should be in such a page.

Thanks a lot
Stephan

[ddmesh]

I found a solution/workaround to get the full list. But this changes the initial format of the resulting json.

in scancodeio/settings.py at the end comment both variables.

#    "DEFAULT_PAGINATION_CLASS": "rest_framework.pagination.PageNumberPagination",
#    "PAGE_SIZE": 10,

Then create the container again.

 docker-compose stop
 docker-compose build
 docker-compose start

Now the Output directly contains the array of projects without the pagination information.

@tdruez perhaps you can add this as an extra endpoint or allow passing perhaps a special value "all" for "&page=".

[tdruez]

Hi @ddmesh see 3340a72 for changes.

The default page_size set to 10 was a bit too conservative, i's now raised to 50.

A new setting is also available if you want to control that value on your ScanCode.io instance, see https://scancodeio.readthedocs.io/en/latest/scancodeio-settings.html#scancodeio-rest-api-page-size
You do not have to edit the scancodeio/settings.py but can now use the .env file.

I hope those changes work or you.

allow passing perhaps a special value "all" for "&page=".

Note that offering such ability on large dataset will have some consequence.
On multi-user instances, we would not want that a user triggers a query returning millions of rows through the REST API.

[ddmesh]

hey, thanks a lot for the fast solution.
this helps.
/Stephan