Git Synch #213

Activity

1. Parakoos

   added

   enhancementNew feature or requestNew feature or request

   on Jul 11, 2024
2. dhalbert commented on Jul 11, 2024

   

   dhalbert

   on Jul 11, 2024

   Contributor

   More actions

   The git repo could live on your host computer drive, and be mirrored by some mechanism to CIRCUITPY. This is more important when CIRCUITPY is small, since a git repo is much larger than the snapshot of the latest commit.

   mpremote, which currently works only on MicroPython (but could be made to work on CircuitPython easily) has a "mounting" mechanism which might work, but I haven't looked at in detail.
3. tannewt commented on Jul 11, 2024

   

   tannewt

   on Jul 11, 2024

   Member

   More actions

   The simplest way to do this may be to add zip support and then fetch the zip from the web periodically. set_next_code_file() can be used to switch to the new version.
4. Parakoos commented on Jul 11, 2024

   

   Parakoos

   on Jul 11, 2024

   Last edited by Parakoos

   Author

   More actions

   The simplest way to do this may be to add zip support and then fetch the zip from the web periodically. set_next_code_file() can be used to switch to the new version.

   That is an interesting option. Is there support for unzipping .zip files in circuitpython? I looked around the library bundle but didn't see anything.

   Another option would be to write a function that:

   1. Turn off auto-reload on file changes
   2. Lists all the files on the MCU
   3. For each (filename), make a request for https://raw.githubusercontent.com/(user)/(repo)/(branch)/(filename)
   4. Download the file content and overwrite the file on the MCU.
   5. supervisor.reload()

   It won't pick up new files, but it would update all existing files. You could then have a trigger (maybe press a button 5 times) to initiate the download from github. (this all assumes you have a wifi board of course)

   What do you think? Would this work? Does CP have access to overwrite its own files?
5. Parakoos commented on Jul 12, 2024

   

   Parakoos

   on Jul 12, 2024

   Author

   More actions

   I put this idea to the test, and it seems to work!

   BEWARE RUNNING THIS CODE! IT WILL OVERWRITE EVERY FILE ON THE MCU THAT IS ALSO ON THE GITHUB REPO BRANCH!

   import storage
   import ssl
   import socketpool
   import wifi
   import time
   import supervisor
   from adafruit_requests import Session
   from os import getenv
   
   GITHUB_USER = getenv("GITHUB_USER")
   GITHUB_REPO = getenv("GITHUB_REPO")
   GITHUB_BRANCH = getenv("GITHUB_BRANCH")
   GITHUB_TOKEN = getenv("GITHUB_TOKEN")
   
   def run():
   	try:
   		storage.remount("/", readonly=False)
   	except RuntimeError as e:
   		print('Failed to remount the file-system to be writeable. Cannot pull data from GitHub without that.')
   		raise e
   
   	pool = socketpool.SocketPool(wifi.radio)
   	ssl_context = ssl.create_default_context()
   	session = Session(pool, ssl_context)
   	header_auth={'Authorization':f"Bearer {GITHUB_TOKEN}"}
   
   	with session.get(f'https://api.github.com/repos/{GITHUB_USER}/{GITHUB_REPO}/branches/{GITHUB_BRANCH}', headers=header_auth) as response:
   		json = response.json()
   		tree_url = json['commit']['commit']['tree']['url']
   
   	with session.get(f"{tree_url}?recursive=1", headers=header_auth) as response:
   		json = response.json()
   		files = json['tree']
   
   	for file in files:
   		if file['type'] == 'blob':
   			print(file['path'])
   			with session.get(file['url'], headers=header_auth|{'accept': 'application/vnd.github.raw+json'}) as response:
   				with open(file['path'], "w") as fp:
   					fp.write(response.text)
   	print('Pulled down the latest data from GitHub! Reloading in 1 second.')
   	time.sleep(1)
   	supervisor.reload()
6. Parakoos commented on Jul 12, 2024

   

   Parakoos

   on Jul 12, 2024

   Author

   More actions

   Oh, and about the github token... You can make a read-only token only (content only) for the repo in question. That way the token isn't so sensitive to put in settings.toml since all it can do is read the repo.

   https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens
7. tannewt commented on Jul 12, 2024

   

   tannewt

   on Jul 12, 2024

   Member

   More actions

   No zip file support yet. I'd be interested in seeing it added though.

   We do have a zlib library that might be able to be used to decompress gzip and then you can write python code to unpack the tar. Looks like repos can be fetched as .tar.gz: https://github.com/adafruit/circuitpython/archive/refs/heads/main.tar.gz