You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
quarto render on shared file storage fails with locked database #13006
After updating quarto to 1.7.31, we run in the following problem:
We have a mounted shared file system on which we get the following error when rendering a qmd file.
ERROR: database is locked: Error code 5: The database file is locked
Stack trace:
at async Object.openKv (ext:deno_kv/01_db.ts:10:15)
at async ProjectCacheImpl.init (file:///opt/quarto/bin/quarto.js:51070:22)
at async createProjectCache (file:///opt/quarto/bin/quarto.js:51197:5)
at async projectContext (file:///opt/quarto/bin/quarto.js:82381:32)
at async render (file:///opt/quarto/bin/quarto.js:91820:19)
at async Command.actionHandler (file:///opt/quarto/bin/quarto.js:92001:32)
at async Command.execute (file:///opt/quarto/bin/quarto.js:8253:13)
at async Command.parseCommand (file:///opt/quarto/bin/quarto.js:8143:20)
at async quarto (file:///opt/quarto/bin/quarto.js:128476:9)
at async file:///opt/quarto/bin/quarto.js:128506:9
if the file is stored in the home directory of a user, the error does not appear.
We did not have issues rendering qmd files on the shared file system before the update of quarto.
I have tried to remove the deno cache but the error remains.
We generally discourage using Quarto in a shared network drives for this reason. In the same way that it's not generally safe eg to use git in a Dropbox, there are parts of Quarto that will not work well in a shared drive because these shared drives behave very differently from file systems. That's not something we can fix, unfortunately.
We have a mounted shared file system on which we get the following error when rendering a qmd file.
Just curious to confirm our thoughts:
Is this a shared drive where you are working on several users at the same time on the same file? Or running several quarto render on the same files?
What exactly is your environment configuration?
Quarto 1.7 uses Deno.kv to have caching of project information, and the Deno.kv mechanism requires locking by Windows, which means only one access.
Quarto is running in a docker container with Ubuntu 22.04.
I'll have to confirm, but the error appears on any new quarto file by any user on the shared drive.
What I can see is that the render process creates ~/.cache/deno/ with *-wal and *-shm files in the respective users home directory. I can't locate such a folder on the shared drive.
I guess I'll work around this in the future, but just out of curiosity:
Is there any way to "unlock" this situation? Also: Were we just lucky that this never happened with quarto < 1.7 or is this due to the introduction of Deno.kv in 1.7?
The ~/.cache/deno dir seems to be for Deno’s internal SQLite usage. That is the default location. It is possible to redirect it by
export DENO_DIR=<path>
I did test that and quarto did indeed write the cache to this location but didnt solve the issue. Could deleting the quarto cache "unlock" the database?
Would you be able to share a Docker container that has the issue? What would we need to reproduce the shared folder setting?
I am afraid that I cannot share any docker containers since they are for internal use only. I will try to derive a minimal repro from it if possible though.
I will try to derive a minimal repro from it if possible though.
That would be helpful. Right now I am quite puzzled by the fact that stack trace shows problem happens in ProjectCacheImpl which should initialize the db using Deno.kv() in the scratch directory (<project root>/.quarto/project-cache
For context, all previous issues we had with Deno.kv and locking were on Windows, as Windows does have a specific locking mechanism that requires explicit unlock.
For example, deleting the cache dir with the DB wouldn't be possible if it was not unlocked beforehand.
Understanding the setting and where the files are really created to cause this type of issue would help us better understand and get to the bottom of this.
Right now, our POV, is shared drive means shared file with multiple access at the same time, which does not play well with Deno.kv it seems.
Awesome. Don't hesitate to reach out (even for quick pairing if this can help). We would like to be sure this is what we think this is, and not some other edge case. Thank you !
@souadg this is not an easy issue. From what we understand, it is currently a limitation with how Deno and SQLite work, and there is a problem with the mounted drive. It is not just Quarto.
What help us is sharing new information if you have or confirm the technical part based on what was shared already. For example, is your drive mounted the same way as the other.
team-shared drives: these are project folders that multiple users can access with permissions, again hosted on a the central server.
Just a reminder that Quarto won't behave without problems if several users are working on the same document at the same time. If two users run quarto render within the same repository on the same file, there will be some problems, and not just the one related to this issue.
@cderv thanks for your feedback on Deno/SQLite limitation.
What help us is sharing new information if you have or confirm the technical part based on what was shared already. For example, is your drive mounted the same way as the other.
We mount drives with mount -t cifs -ousername={username},password="{password}",dir_mode=0750,file_mode=0740,gid={uid},uid={uid},sec=ntlmv2i {source} {destination}
Just a reminder that Quarto won't behave without problems if several users are working on the same document at the same time. If two users run quarto render within the same repository on the same file, there will be some problems, and not just the one related to this issue.
Thanks for the reminder. There is currentyly no issue of multiple users rendering quarto projects at the same time. We encounter this issue with ERROR: database is locked: Error code 5: The database file is locked when we work in personal home mounted drives.
There is currentyly no issue of multiple users rendering quarto projects at the same time
I think things could happen silently. What I want to warn you is that quarto and computation tool like knitr are designed to work on document name, in a single folder for a single render.
As soon as you start parallelization from the same files, there could be conflict in the way that a render can create some resources that another render at the same time may use, delete, replace, etc...
There could be issues if a render expect a resource already deleted. But otherwise, it could just a one render modifying content from a previous render before it finishes.
Anyhow, you are aware of the behavior. Two 'quarto render' processes working at the same time on the same files could step over each other. This will be the same for two users working on a same files at the same time.
For this database lock issue, I'll find a solution.
Anyhow, you are aware of the behavior. Two 'quarto render' processes working at the same time on the same files could step over each other. This will be the same for two users working on a same files at the same time.
This is useful information, duly noted and will take this into account.
For this database lock issue, I'll find a solution.
I've had the same problem: quarto fails when you are using a docker image of rstudio and the file is on a mounted network drive.
A hack is to use a tempdir() and copy over the files. It does render a html file without error codes. Although I've not yet managed to get a preview with the expected stylistic aspects, just a vanilla plain html view.
Activity
cscheid commentedon Jun 30, 2025
We generally discourage using Quarto in a shared network drives for this reason. In the same way that it's not generally safe eg to use git in a Dropbox, there are parts of Quarto that will not work well in a shared drive because these shared drives behave very differently from file systems. That's not something we can fix, unfortunately.
cderv commentedon Jun 30, 2025
Just curious to confirm our thoughts:
Is this a shared drive where you are working on several users at the same time on the same file? Or running several
quarto renderon the same files?What exactly is your environment configuration?
Quarto 1.7 uses Deno.kv to have caching of project information, and the Deno.kv mechanism requires locking by Windows, which means only one access.
schochastics commentedon Jun 30, 2025
Quarto is running in a docker container with Ubuntu 22.04.
I'll have to confirm, but the error appears on any new quarto file by any user on the shared drive.
What I can see is that the render process creates
~/.cache/deno/with*-waland*-shmfiles in the respective users home directory. I can't locate such a folder on the shared drive.I guess I'll work around this in the future, but just out of curiosity:
Is there any way to "unlock" this situation? Also: Were we just lucky that this never happened with quarto < 1.7 or is this due to the introduction of Deno.kv in 1.7?
cderv commentedon Jun 30, 2025
I do believe this is the introduction of using Deno.kv for caching. This is what requires locking and unlocking.
I am curious about this path. Our
createProjectCache()should targetproject-cacheinside.quartofolder in a Quarto project. 🤔Quarto do use
~/.cachewhen created fromquarto-cli/src/core/appdirs.ts
Lines 140 to 144 in 85b8193
which is used for User cache directory on Linux
quarto-cli/src/core/appdirs.ts
Lines 74 to 90 in 85b8193
However, I am curious about the
deno/part, as Quarto should be usingquarto/quarto-cli/src/core/appdirs.ts
Lines 19 to 21 in 85b8193
Would you be able to share a Docker container that has the issue? What would we need to reproduce the shared folder setting?
I'll let Calors add his insights on this.
schochastics commentedon Jun 30, 2025
The
~/.cache/denodir seems to be for Deno’s internal SQLite usage. That is the default location. It is possible to redirect it byI did test that and quarto did indeed write the cache to this location but didnt solve the issue. Could deleting the quarto cache "unlock" the database?
I am afraid that I cannot share any docker containers since they are for internal use only. I will try to derive a minimal repro from it if possible though.
cderv commentedon Jul 1, 2025
That would be helpful. Right now I am quite puzzled by the fact that stack trace shows problem happens in
ProjectCacheImplwhich should initialize the db using Deno.kv() in the scratch directory (<project root>/.quarto/project-cachequarto-cli/src/core/cache/cache.ts
Lines 69 to 72 in 85b8193
For context, all previous issues we had with Deno.kv and locking were on Windows, as Windows does have a specific locking mechanism that requires explicit unlock.
For example, deleting the cache dir with the DB wouldn't be possible if it was not unlocked beforehand.
Understanding the setting and where the files are really created to cause this type of issue would help us better understand and get to the bottom of this.
Right now, our POV, is shared drive means shared file with multiple access at the same time, which does not play well with Deno.kv it seems.
Thanks a lot.
schochastics commentedon Jul 1, 2025
No worries. It might take some time on my end but since I would like to understand this issue better too, I'll try to create a minimal repro example.
cderv commentedon Jul 1, 2025
Awesome. Don't hesitate to reach out (even for quick pairing if this can help). We would like to be sure this is what we think this is, and not some other edge case. Thank you !
github-actions commentedon Jul 16, 2025
25 remaining items
cderv commentedon Jan 7, 2026
@souadg this is not an easy issue. From what we understand, it is currently a limitation with how Deno and SQLite work, and there is a problem with the mounted drive. It is not just Quarto.
What help us is sharing new information if you have or confirm the technical part based on what was shared already. For example, is your drive mounted the same way as the other.
Just a reminder that Quarto won't behave without problems if several users are working on the same document at the same time. If two users run
quarto renderwithin the same repository on the same file, there will be some problems, and not just the one related to this issue.souadg commentedon Jan 8, 2026
@cderv thanks for your feedback on Deno/SQLite limitation.
We mount drives with mount
-t cifs -ousername={username},password="{password}",dir_mode=0750,file_mode=0740,gid={uid},uid={uid},sec=ntlmv2i {source} {destination}Thanks for the reminder. There is currentyly no issue of multiple users rendering quarto projects at the same time. We encounter this issue with
ERROR: database is locked: Error code 5: The database file is lockedwhen we work in personal home mounted drives.cderv commentedon Jan 8, 2026
Thanks for the details.
I think things could happen silently. What I want to warn you is that quarto and computation tool like knitr are designed to work on document name, in a single folder for a single render.
As soon as you start parallelization from the same files, there could be conflict in the way that a render can create some resources that another render at the same time may use, delete, replace, etc...
There could be issues if a render expect a resource already deleted. But otherwise, it could just a one render modifying content from a previous render before it finishes.
Anyhow, you are aware of the behavior. Two 'quarto render' processes working at the same time on the same files could step over each other. This will be the same for two users working on a same files at the same time.
For this database lock issue, I'll find a solution.
souadg commentedon Jan 8, 2026
@cderv
This is useful information, duly noted and will take this into account.
That would be fab, merci Christophe !
shug0131 commentedon Jun 12, 2026
I've had the same problem: quarto fails when you are using a docker image of rstudio and the file is on a mounted network drive.
A hack is to use a tempdir() and copy over the files. It does render a html file without error codes. Although I've not yet managed to get a preview with the expected stylistic aspects, just a vanilla plain html view.
Force VUEGEN onto local scratch disk to work around a Quarto bug
Force VUEGEN onto local scratch disk to work around a Quarto bug
Get netflow pipeline to run in Seqera (#60)