Skip to content

quarto render on shared file storage fails with locked database #13006

Open

Description

@schochastics

After updating quarto to 1.7.31, we run in the following problem:
We have a mounted shared file system on which we get the following error when rendering a qmd file.

ERROR: database is locked: Error code 5: The database file is locked

Stack trace:
    at async Object.openKv (ext:deno_kv/01_db.ts:10:15)
    at async ProjectCacheImpl.init (file:///opt/quarto/bin/quarto.js:51070:22)
    at async createProjectCache (file:///opt/quarto/bin/quarto.js:51197:5)
    at async projectContext (file:///opt/quarto/bin/quarto.js:82381:32)
    at async render (file:///opt/quarto/bin/quarto.js:91820:19)
    at async Command.actionHandler (file:///opt/quarto/bin/quarto.js:92001:32)
    at async Command.execute (file:///opt/quarto/bin/quarto.js:8253:13)
    at async Command.parseCommand (file:///opt/quarto/bin/quarto.js:8143:20)
    at async quarto (file:///opt/quarto/bin/quarto.js:128476:9)
    at async file:///opt/quarto/bin/quarto.js:128506:9

if the file is stored in the home directory of a user, the error does not appear.
We did not have issues rendering qmd files on the shared file system before the update of quarto.

I have tried to remove the deno cache but the error remains.

Activity

  1. self-assigned this
    on Jun 30, 2025
  2. cscheid commented on Jun 30, 2025

    @cscheid
    Member

    We generally discourage using Quarto in a shared network drives for this reason. In the same way that it's not generally safe eg to use git in a Dropbox, there are parts of Quarto that will not work well in a shared drive because these shared drives behave very differently from file systems. That's not something we can fix, unfortunately.

  3. added this to the Future milestone on Jun 30, 2025
  4. cderv commented on Jun 30, 2025

    @cderv
    Member

    We have a mounted shared file system on which we get the following error when rendering a qmd file.

    Just curious to confirm our thoughts:
    Is this a shared drive where you are working on several users at the same time on the same file? Or running several quarto render on the same files?

    What exactly is your environment configuration?

    Quarto 1.7 uses Deno.kv to have caching of project information, and the Deno.kv mechanism requires locking by Windows, which means only one access.

  5. schochastics commented on Jun 30, 2025

    @schochastics
    Author

    Quarto is running in a docker container with Ubuntu 22.04.
    I'll have to confirm, but the error appears on any new quarto file by any user on the shared drive.

    What I can see is that the render process creates ~/.cache/deno/ with *-wal and *-shm files in the respective users home directory. I can't locate such a folder on the shared drive.

    I guess I'll work around this in the future, but just out of curiosity:
    Is there any way to "unlock" this situation? Also: Were we just lucky that this never happened with quarto < 1.7 or is this due to the introduction of Deno.kv in 1.7?

  6. cderv commented on Jun 30, 2025

    @cderv
    Member

    Were we just lucky that this never happened with quarto < 1.7 or is this due to the introduction of Deno.kv in 1.7?

    I do believe this is the introduction of using Deno.kv for caching. This is what requires locking and unlocking.

    What I can see is that the render process creates ~/.cache/deno/

    I am curious about this path. Our createProjectCache() should target project-cache inside .quarto folder in a Quarto project. 🤔

    Quarto do use ~/.cache when created from

    function xdgUserCacheDir(appName: string) {
    const cacheHome = Deno.env.get("XDG_CACHE_HOME") ||
    join(Deno.env.get("HOME") || "", ".cache");
    return join(cacheHome, appName);
    }

    which is used for User cache directory on Linux

    export function userCacheDir(appName: string) {
    switch (platformOs) {
    case "darwin":
    return darwinUserCacheDir(appName);
    case "windows":
    return windowsUserDataDir(appName);
    default: // assume generic unix
    // in 1.32.5 this is:
    // case "linux":
    // case "netbsd":
    // case "aix":
    // case "solaris":
    // case "illumos":
    // case "freebsd":
    return xdgUserCacheDir(appName);
    }
    }

    However, I am curious about the deno/ part, as Quarto should be using quarto/

    export function quartoCacheDir(subdir?: string) {
    return quartoDir(userCacheDir, subdir);
    }

    Quarto is running in a docker container with Ubuntu 22.04.

    Would you be able to share a Docker container that has the issue? What would we need to reproduce the shared folder setting?

    I'll let Calors add his insights on this.

  7. added
    needs-reproIssues that are blocked until reporter provides an adequate reproduction
    on Jun 30, 2025
  8. schochastics commented on Jun 30, 2025

    @schochastics
    Author

    The ~/.cache/deno dir seems to be for Deno’s internal SQLite usage. That is the default location. It is possible to redirect it by

    export DENO_DIR=<path>

    I did test that and quarto did indeed write the cache to this location but didnt solve the issue. Could deleting the quarto cache "unlock" the database?

    Would you be able to share a Docker container that has the issue? What would we need to reproduce the shared folder setting?

    I am afraid that I cannot share any docker containers since they are for internal use only. I will try to derive a minimal repro from it if possible though.

  9. cderv commented on Jul 1, 2025

    @cderv
    Member

    I will try to derive a minimal repro from it if possible though.

    That would be helpful. Right now I am quite puzzled by the fact that stack trace shows problem happens in ProjectCacheImpl which should initialize the db using Deno.kv() in the scratch directory (<project root>/.quarto/project-cache

    async init(): Promise<void> {
    const indexPath = join(this.projectScratchDir, "project-cache");
    ensureDirSync(indexPath);
    this.index = await Deno.openKv(join(indexPath, "deno-kv-file"));

    For context, all previous issues we had with Deno.kv and locking were on Windows, as Windows does have a specific locking mechanism that requires explicit unlock.
    For example, deleting the cache dir with the DB wouldn't be possible if it was not unlocked beforehand.

    Understanding the setting and where the files are really created to cause this type of issue would help us better understand and get to the bottom of this.

    Right now, our POV, is shared drive means shared file with multiple access at the same time, which does not play well with Deno.kv it seems.

    Thanks a lot.

  10. schochastics commented on Jul 1, 2025

    @schochastics
    Author

    No worries. It might take some time on my end but since I would like to understand this issue better too, I'll try to create a minimal repro example.

  11. cderv commented on Jul 1, 2025

    @cderv
    Member

    Awesome. Don't hesitate to reach out (even for quick pairing if this can help). We would like to be sure this is what we think this is, and not some other edge case. Thank you !

  12. github-actions commented on Jul 16, 2025

    @github-actions
  13. added
    staleIssues open for 30+ days without providing a "working" reproducible example
    on Jul 16, 2025
  14. 25 remaining items

  15. cderv commented on Jan 7, 2026

    @cderv
    Member

    @souadg this is not an easy issue. From what we understand, it is currently a limitation with how Deno and SQLite work, and there is a problem with the mounted drive. It is not just Quarto.

    What help us is sharing new information if you have or confirm the technical part based on what was shared already. For example, is your drive mounted the same way as the other.

    team-shared drives: these are project folders that multiple users can access with permissions, again hosted on a the central server.

    Just a reminder that Quarto won't behave without problems if several users are working on the same document at the same time. If two users run quarto render within the same repository on the same file, there will be some problems, and not just the one related to this issue.

  16. souadg commented on Jan 8, 2026

    @souadg

    @cderv thanks for your feedback on Deno/SQLite limitation.

    What help us is sharing new information if you have or confirm the technical part based on what was shared already. For example, is your drive mounted the same way as the other.

    We mount drives with mount
    -t cifs -ousername={username},password="{password}",dir_mode=0750,file_mode=0740,gid={uid},uid={uid},sec=ntlmv2i {source} {destination}

    Just a reminder that Quarto won't behave without problems if several users are working on the same document at the same time. If two users run quarto render within the same repository on the same file, there will be some problems, and not just the one related to this issue.

    Thanks for the reminder. There is currentyly no issue of multiple users rendering quarto projects at the same time. We encounter this issue with ERROR: database is locked: Error code 5: The database file is locked when we work in personal home mounted drives.

  17. cderv commented on Jan 8, 2026

    @cderv
    Member

    Thanks for the details.

    There is currentyly no issue of multiple users rendering quarto projects at the same time

    I think things could happen silently. What I want to warn you is that quarto and computation tool like knitr are designed to work on document name, in a single folder for a single render.

    As soon as you start parallelization from the same files, there could be conflict in the way that a render can create some resources that another render at the same time may use, delete, replace, etc...
    There could be issues if a render expect a resource already deleted. But otherwise, it could just a one render modifying content from a previous render before it finishes.

    Anyhow, you are aware of the behavior. Two 'quarto render' processes working at the same time on the same files could step over each other. This will be the same for two users working on a same files at the same time.

    For this database lock issue, I'll find a solution.

  18. souadg commented on Jan 8, 2026

    @souadg

    @cderv

    Anyhow, you are aware of the behavior. Two 'quarto render' processes working at the same time on the same files could step over each other. This will be the same for two users working on a same files at the same time.

    This is useful information, duly noted and will take this into account.

    For this database lock issue, I'll find a solution.

    That would be fab, merci Christophe !

  19. modified the milestones: v1.9, v1.10 on Feb 24, 2026
  20. shug0131 commented on Jun 12, 2026

    @shug0131

    I've had the same problem: quarto fails when you are using a docker image of rstudio and the file is on a mounted network drive.

    A hack is to use a tempdir() and copy over the files. It does render a html file without error codes. Although I've not yet managed to get a preview with the expected stylistic aspects, just a vanilla plain html view.

    tdir <- tempdir()
    file.copy(from="posts", to=tdir, recursive=TRUE)
    quarto::quarto_render(input=file.path(tdir, "posts/my_blog_file.qmd"))
    list.files(tdir)
    
  21. modified the milestones: v1.10, v1.11 on Jul 30, 2026
  22. modified the milestones: v1.11, v1.12 on Oct 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bugSomething isn't workingfile-systemsissues with NFS, google drive, onedrive, etc

Type

No type

Projects

No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions