Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

resource_data page gives HTTP 504 timeout on very large resources #187

Open
ThrawnCA opened this issue May 18, 2023 · 0 comments
Open

resource_data page gives HTTP 504 timeout on very large resources #187

ThrawnCA opened this issue May 18, 2023 · 0 comments

Comments

@ThrawnCA
Copy link
Collaborator

The resource_data page doesn't paginate the log entries that it's displaying, so very large resources (eg https://www.data.qld.gov.au/dataset/queensland-covid-19-case-line-list-age-groups/resource/a14317b7-2fca-41b7-8294-9a1f7a085b0f with 1.7 million records) will result in excessively long page load times (our system times out at 30 seconds).
ThrawnCA added a commit to qld-gov-au/ckanext-xloader that referenced this issue May 19, 2023
@ThrawnCA
hide excessive numbers of resource_data log entries, ckan#187
4111f3f 
- Default to showing the first 50 and last 50 entries, with links to display more
- This dramatically improves page load times for very large resources
ThrawnCA added a commit that referenced this issue Jan 29, 2024
@ThrawnCA
Merge pull request #188 from qld-gov-au/resource_data-timeout-187
4f3864f 
Hide excessive numbers of resource_data log entries, #187
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@ThrawnCA