Skip to content

[cloud] emulate user's local directory structure in VM - #440

Merged
savil merged 3 commits into
mainfrom
savil/vm-folder
Jan 10, 2023
Merged

savil merged 3 commits into
mainfrom
savil/vm-folder

Conversation

@savil

@savil savil commented Jan 7, 2023 •

Copy link
Copy Markdown
Collaborator

Summary

Problem
A scenario we want to protect against is two distinct projects sharing the same directory name. For example, consider projects located at ~/customer1/my_project and ~/customer2/my_project. Prior to this PR, we would attempt to sync both projects to the same folder in the VM: ~/code/my_project.

Approach
This PR attempts to solve this conflict by seeking to emulate the user's directory structure in the VM. So, for the aforementioned example projects, they will sync to ~/customer1/my_project and ~/customer2/my_project in the VM as well. This has the additional benefit that if a user has relative paths between two projects they are actively developing (for example, a golang library and golang application that uses that library), then these relative paths will continue working in the cloud-shell, in accordance with our desires to make the cloud-shell experience akin to local shells.

Edge case: projects outside homedir
An edge case is for projects that are located locally outside the user's homedir. For security reasons, in the VM, we do not want to give the user root access, or access to folders outside their homedir. We choose to sync to a special folder called: ~/outside-homedir-code/<absolute path>. For example, a project located locally at /my_company/my_project will sync to ~/outside-homedir-code/my_company/my_project. There are two downsides to this which are tolerable IMO:

  1. If a user also has a local folder called ~/devbox-cloud and they want to sync those projects then there will be a conflict.
  2. Relative paths between a project in homedir and outside the homedir will not work. The user can inspect the paths in the cloud-shells via pwd and update the relative paths to match that.

How was it tested?

for testdata/go/go-1.19: could sync to VM via devbox cloud shell, and inspect directory in VM with pwd.

for edge case: projects outside homedir:

  1. copied examples/testdata/rust/rust-stable to /usr/local/testy-nonhome-dir/rust-stable
  2. fixed permissions: sudo chown savil:staff /usr/local/testy-nonhome-dir/rust-stable
  3. devbox cloud shell to VM
  4. pwd in VM gave: /home/savil/outside-homedir-code/usr/local/testy-nonhome-dir/rust-stable
  5. devbox add gcc and then cargo run ran successfully.

savil commented Jan 7, 2023

Copy link
Copy Markdown
Collaborator Author

Current dependencies on/for this PR:

This comment was auto-generated by Graphite.

@savil
savil force-pushed the savil/vm-folder branch 2 times, most recently from d7c5836 to da1c8ee Compare January 9, 2023 23:40
@savil
savil requested review from LucilleH and gcurtis January 9, 2023 23:43
@savil
savil marked this pull request as ready for review January 9, 2023 23:43
Comment thread internal/cloud/cloud.go Outdated

// In a background routine, update the sync status in the cloud VM
go updateSyncStatus(mutagenSessionName, username, hostname, projectName)
go updateSyncStatus(mutagenSessionName, username, hostname, filepath.Base(projectPath))

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be hyphenatePath(projectPath)? Would this status also have conflict?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh yeah! good call. Updated.

@savil
savil force-pushed the savil/vm-folder branch 2 times, most recently from 52eded7 to ad94d48 Compare January 10, 2023 01:49
@savil

savil commented Jan 10, 2023

Copy link
Copy Markdown
Collaborator Author
  1. Made an update that renames functions to relativeProjectPathInVM and absoluteProjectPathInVM. This feels more clear to me. I need to re-test after this change.
  2. Added Lucille's suggestion to hyphenate the project path for the starship prompt updater.
  3. This PR is dependent on the VM update in https://github.com/jetpack-io/axiom/pull/2934 (closed source, sorry). I need to make that change backwards compatible, and then deploy it prior to landing this code.

@savil
savil merged commit 09b7200 into main Jan 10, 2023
@savil
savil deleted the savil/vm-folder branch January 10, 2023 19:55
mikeland73 added a commit that referenced this pull request Sep 16, 2026
…de-extension (#2980)

## Summary

Follow-up to #2979. Clears the last **10 open Dependabot alerts** (8
medium, 2 low) — the repo now has zero open alerts. Each manifest was
re-locked using the toolchain its own `devbox.json` provides (via
`devbox shellenv`), then every package in every touched lockfile was
audited against [OSV](https://osv.dev): **283 unique packages, 0
advisories**.

| Location | Change | Alerts cleared |
| --- | --- | --- |
| `examples/development/python/poetry/poetry-demo` | pytest `7.4.4` →
`9.1.1`; python constraint `^3.8` → `^3.10` | #438, #368 |
| `examples/development/python/poetry/poetry-pyproject-subdir/service` |
pytest `7.4.4` → `9.1.1`; python constraint `^3.8` → `^3.10` | #439,
#367 |
| `examples/development/python/pipenv` | lock-only: pytest → `9.1.1`
(grpcio `1.84.0`, etc. moved along) | #366 |
| `examples/data_science/pytorch/basic-example` | lock-only: torch
`2.7.1` → `2.14.0`, setuptools `80.10.2` → `84.0.0` (transitive) | #453,
#440, #403, #402 |
| `vscode-extension` | lock-only: serialize-javascript `7.0.4` → `7.1.1`
| #383 |

### Notes

**pytest needed the Python floor raised.** The tmpdir fix
(GHSA-6w46-j5rx-g56g) only exists in 9.0.3+, and pytest 9 requires
Python ≥ 3.10, so the two poetry examples' `python = "^3.8"` constraint
had to move to `^3.10` for the fix to be reachable. Both examples'
`devbox.json` already install `python@latest`, so nothing changes for
anyone running them through devbox. `pytest = "^7.2.2"` → `"^9.0.3"` is
the only other manifest edit in the PR.

**torch moved to CUDA 13 wheels.** `torch = "^2.7.0"` already admitted
2.14.0, so `pyproject.toml` is untouched, but the Linux extras in the
lock shifted from `nvidia-*-cu12` to CUDA 13 packages (`cuda-toolkit
13.0.3`, `nvidia-cudnn-cu13`, etc.). The wheels bundle their own
runtime, so this matters only for driver version on Linux hosts (CUDA 13
needs a 580+ driver). The nix `cudatoolkit` pinned in that example's
`devbox.lock` is 11.7 from an old nixpkgs and was already mismatched
with the previous cu12 wheels — I left it alone as it's unrelated to the
advisories.

**Pre-existing, not fixed here:** the pytorch example's `poetry install`
fails on main because `pyproject.toml` declares `packages = [{include =
"devbox_cuda_dev"}]` and that directory doesn't exist. `poetry install
--no-root` works; I verified torch 2.14.0 imports and runs on CPU that
way.

**vscode-extension:** the `resolutions` entry already allowed `^7.0.0`,
so only `yarn.lock` moved. While there, `yarn audit` flagged `ajv
6.12.6` (GHSA-2g4f-4pwh-qvx6) and `diff 5.2.0` (GHSA-73rr-hh4g-fpgx);
both patches fall inside existing ranges so they were refreshed too.
`yarn audit --level low` is now clean.

## How was it tested?

- `devbox run test` (poetry-demo) and `devbox run run_test`
(poetry-pyproject-subdir): 1 passed each.
- `devbox run run_test` (pipenv): runs `main.py` successfully.
- pytorch example: `poetry install --no-root` + import/matmul smoke test
→ `torch 2.14.0, numpy 1.26.4, setuptools 84.0.0`.
- vscode-extension: `yarn install --frozen-lockfile`, `yarn compile`,
`yarn lint`, `yarn audit --level low` → 0 vulnerabilities.
- OSV batch query over every package in all five lockfiles → 0
advisories.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

2 participants