You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Integrate MacOS DMG and Mac App Store builds into Nix Build Pipeline #10
Can it work if Nix is running on MacOS? Need to experiment with this on Mac system. We have the M1 mac system to test on and also Mac systems that @gideonairex is running on.
If it works, we can at least have a consistent platform to work on, using Nix on NixOS and using Nix on MacOS.
However I have heard from @rbellec that Nix has some problems with the latest OS BigSur. So we should probably avoid that.
After some experimenting, this is what we've found. the @electron-forge/maker-dmg package creates issues when trying to run nix-build as there exists an optional dependency that cannot be built on the nix-build environment. This is because node2nix does not differentiate between optional and regular dependencies, and therefore will not skip the dependency that can normally be skipped in a userland npm install.
I have tried adding --strip-optional-dependencies to node2nix however, this strips app optional dependencies and therefore cause issues with other builds, e.g. the deb builds do not work anymore.
Attempting to substituteInPlace the package.json also does not give any results, as:
If we don't include package-lock.json, node2nix attempts to generate it, which requires fetching from online registries, which cannot be done during a nix build as it must remain offline.
Attempting to recreate it using npm install leads to the same problem
Replacing all affected lines in package-lock.json is too messy a process to consider as we need to consider blocks of code and several subdependencies.
As a result, for now in the source, we will not include @electron-forge/maker-dmg. It can STILL BE INSTALLED and used in userland, in the nix-shell.
For now, anytime we have this situation, either we repackage that as Nix dependency if it provides an executable, or we just have to interactively do it with npm install ... and then not commit that to the master. Or maintain it as a separate branch.
There's definitely a way to ignore certain dependencies once we have generated them. But we will wait until we have MacOS CI/CD available before experimenting on this. We'll need to consider the usage of our application key and a manual Q&A process for our Mac applications.
Activity
CMCDragonkai commentedon Mar 11, 2021
Also we have manual code signing that we have to do. How to integrate this? I'm not sure of the workflow. @gideonairex can you describe it?
[-]Deal with DMG and MAS builds[/-][+]Regarding DMG and MAS builds[/+]DrFacepalm commentedon Mar 12, 2021
After some experimenting, this is what we've found. the
@electron-forge/maker-dmgpackage creates issues when trying to runnix-buildas there exists an optional dependency that cannot be built on thenix-buildenvironment. This is becausenode2nixdoes not differentiate between optional and regular dependencies, and therefore will not skip the dependency that can normally be skipped in a userlandnpm install.I have tried adding
--strip-optional-dependenciestonode2nixhowever, this strips app optional dependencies and therefore cause issues with other builds, e.g. the deb builds do not work anymore.Attempting to
substituteInPlacethepackage.jsonalso does not give any results, as:package-lock.json,node2nixattempts to generate it, which requires fetching from online registries, which cannot be done during a nix build as it must remain offline.npm installleads to the same problempackage-lock.jsonis too messy a process to consider as we need to consider blocks of code and several subdependencies.As a result, for now in the source, we will not include
@electron-forge/maker-dmg. It can STILL BE INSTALLED and used in userland, in thenix-shell.CMCDragonkai commentedon Mar 17, 2021
This requires some upstream fix then: svanderburg/node2nix#229.
For now, anytime we have this situation, either we repackage that as Nix dependency if it provides an executable, or we just have to interactively do it with
npm install ...and then not commit that to the master. Or maintain it as a separate branch.[-]Regarding DMG and MAS builds[/-][+]Integrate MacOS DMG and Mac App Store builds into Nix Build Pipeline[/+]joshuakarp commentedon Aug 4, 2021
This is also related to the integration of macOS CI/CD MatrixAI/Polykey#4
CMCDragonkai commentedon Aug 29, 2021
This might be worth exploring: #16 (comment)
Also we did do some overrides of Nix and NPM packages here: https://gitlab.com/MatrixAI/Engineering/nixpkgs-overlay/-/merge_requests/15. This shows you how to override the node packages structure in Nixpkgs upstream. And therefore how to apply an override at the end.
There's definitely a way to ignore certain dependencies once we have generated them. But we will wait until we have MacOS CI/CD available before experimenting on this. We'll need to consider the usage of our application key and a manual Q&A process for our Mac applications.