Repository navigation
connection hangs randomly #204
Description
Hi,
At random times during my notebook session, a sql statement like %sql select count(*) from table will just hang even though the calculation should be fast. This hang cannot be interrupted by jupyter, so I always have to restart the kernel. I can confirm that my postgresql server is still running and I can run the same command when I log in through a client, so this should not be a server side problem.
Wondering if anyone else has seen this and if there's a workaround or solution.
Activity
idomic commentedon Dec 23, 2022
@princesscrypto1337 does it still happen to you (I know this issue is pretty old)
earlev4 commentedon Feb 3, 2023
Huge thanks and appreciation to the contributors of the
ipython-sqlproject! Very grateful for the project!Hi @idomic. This problem is occurring for me. As @princesscrypto1337 mentions, the problem is random. Usually occurs after a lot of interaction with the Jupyter notebook. Like the issue reported, I am also using
%sql select count(*). It also occurs in%sql create or replace table.... The issue seems to happen if I run those statements a couple or few times. In my case, I am usingDuckDB. At first, I was usingJupySQLand I thought it was aDuckDBissue. However, after troubleshooting the issue, and eliminating%sqland using a directcon.execute()the issue was no longer persistent. Just for context, usingcon.execute()theselect count(*)only takes around 40-50 seconds. Letting%sql select count(*)run, can run for hours until stopped. I uninstalledJupySQLand triedipython-sql(version:0.4.1) and the issue appeared again (once again random). When the progress bar fromDuckDBis available, the progress bar usually indicates the hang occurs at different times (32%, 38%, 41%, 88%, 90%, etc.). When the progress bar is not available, I just get the*indicating the code cell is currently executing. In my situation, I can interrupt the kernel. After interrupting the kernel, I tried%reload_ext sqland the problem still persists. Restarting the kernel, I can usually useipython-sqlagain for a%sql select * from table limit 5, but then the issue starts to happen again when I do%sql select count(*). For now, I have suspended use. Thanks for your interest!earlev4 commentedon Feb 7, 2023
Hi. After exhaustive trial and error, the
DuckDBproblem previously mentioned does not appear to be related to theipython-sqlorJupySQLextensions. The issue with the cell hanging in Jupyter occurs when theDuckDBconfiguration isSET enable_progress_bar=true;. There appears to be an issue already already opened in the DuckDB repository - 5407.Sorry to confuse the issue reported by @princesscrypto1337 for
PostgreSQL. However, if someone is encountering cell hangs in Jupyter with DuckDB, try disabling the progress bar until the issue is resolved. Thanks!idomic commentedon Feb 7, 2023
@edublancas
edublancas commentedon Feb 7, 2023
The main issue here is to reproduce the problem. However, I think it'd still be valuable to provide a workaround, perhaps a timeout? So that cells that take too long abort, and there isn't a need to restart the kernel.
Would this help? @princesscrypto1337
mysql tutorial (catherinedevlin#204)