Skip to content

Polykey naming and repository structure #426

Closed
Closed
Polykey naming and repository structure#426
Assignees
CMCDragonkai
Labels
documentationImprovements or additions to documentationepicBig issue with multiple subissuesprocedureAction that must be executed

Description

@CMCDragonkai
CMCDragonkai
opened on Sep 23, 2021

Move to new repo structure:

Polykey

  • js-polykey renames to Polykey
  • keeps all issues in js-polykey
  • license: GPL3 (possible move to APL3)
  • npm: polykey
  • not hosted on nix (it's not an application)
  • supports network API
  • embeddable as a library
  • bin/ code is extracted to Polykey-CLI
  • maintains wiki: theory, reference, guides, tutorials
  • maintains community discussions
  • main README.md (open source entry point)
  • requires cross-platform gitlab runner for building and QA of possibly-native addons

Polykey-CLI

  • license: GPL3
  • npm: polykey-cli
  • nix: polykey-cli
  • extracts bin/ code out of js-polykey
  • requires cross-platform gitlab runner for building and QA

Polykey-Desktop - electron/tauri wrapper around library code

  • license: GPL3
  • not hosted on npm (it's not a JS library or application)
  • nix: polykey-desktop
  • requires cross-platform gitlab runner for building & QA

Polykey-Mobile - nativescript wrapper around library code

  • license: unknown
  • not hosted on npm (it's not a JS library or application)
  • not hosted on nix (it's not a desktop application)
  • requires cross-platform gitlab runner for building & QA

Old spec:

I reckon the js-polykey codebase can be moved into Polykey. There aren't going to be other language implementations of js-polykey. And the desktop application is unlikely to be distributed in NPM anyway. So we can just use the name NPM package name polykey.

This would mean:

  • Polykey - CLI & library code, and all the wiki: theory, reference, guides, tutorials, README.md should be mentioning Polykey-Desktop and Polykey-Mobile
  • Polykey-Desktop - Electron app
  • Polykey-Mobile - Mobile App

We have 3 names on npm:

I think we should revive the name polykey and make use of it. It's less characters to type. We already control it, we just need to undeprecate the name: https://docs.npmjs.com/deprecating-and-undeprecating-packages-or-package-versions. Then that means deprecating @matrixai/polykey in favour of polykey. Neither desktop app nor mobile app would be distributed on NPM so their names can be just polykey-desktop and polykey-mobile in package.json.

In other package repositories, polykey and polykey-desktop would be separate names. So nixpkgs would have polykey and polykey-desktop.

Furthermore we would have a polykey-enterprise a mirror/fork of the repo. Need to study how this is done in gitlab. This version is proprietary or it may be semi-opensource with BSL. See: https://about.gitlab.com/install/ce-or-ee/

We also want to preserve all the commit history of js-polykey, whereas this repo's history is less useful. We could just delete this repository and rename js-polykey to Polykey. But a safer option is probably history merging: https://stackoverflow.com/a/10548919/582917

Also in order to avoid needing to move all of our issues in js-polykey, we would go from Polykey to js-polykey.

Tasks

  1. - Attempt history merging in https://stackoverflow.com/a/10548919/582917, by merging Polykey into js-polykey
  2. - Move all wiki and issues from Polykey to js-polykey, thus superseding this MatrixAI/Polykey.old#3
  3. - Archive and schedule deletion this repository Polykey.old
  4. - Rename js-polykey to "Polykey" for its repository
  5. - Rename package.json for js-polykey to polykey
  6. [ ] - Undeprecate the NPM polykey
  7. [ ] - Publish to polykey
  8. [ ] - Deprecate @matrixai/polykey
  9. [ ] - Change name of Polykey-Desktop package.json to use polykey-desktop - desktop is now a JS application nor library, useless to do npm install. There is no main here. I guess it could work though, since electron is executable from JS, you just end up distributing native binaries... either prebuilds, or some sort of packaging situation where you select the appropriate release. At any case it's not really relevant.

Activity

  1. CMCDragonkai
    added
    procedureAction that must be executed
    on Sep 23, 2021

  2. CMCDragonkai commented on Sep 23, 2021

    @CMCDragonkai
    CMCDragonkai
    on Sep 23, 2021
    MemberAuthor

    I noticed alot of wiki assets were stored in the repo in order to do MatrixAI/Polykey-Docs#5. @emmacasolin

    Those assets are better off being embedded into the wiki repo itself rather than the code repo.

    Instructions to do this are here: https://gist.github.com/TT--/14260aef6c0e31fca5b37e7cb3c53020 and https://gist.github.com/subfuzion/0d3f19c4f780a7d75ba2

    I've done this before... so I will need to write a guide. This requires cloning the wiki git repo itself.

  3. CMCDragonkai
    self-assigned this
    on Sep 30, 2021

  4. CMCDragonkai commented on Oct 5, 2021

    @CMCDragonkai
    CMCDragonkai
    on Oct 5, 2021
    MemberAuthor

    Note that internally we still have a polykey-design repository that can contain all the design materials. Although majority of its information should be ported to the wiki.

    However branding related information still stays in polykey-design. Research notes can stay there too or be part of issues.

  5. CMCDragonkai
    added
    documentationImprovements or additions to documentation
    on Oct 18, 2021

  6. CMCDragonkai commented on Oct 18, 2021

    @CMCDragonkai
    CMCDragonkai
    on Oct 18, 2021
    MemberAuthor

    Note that we should also maintain understanding of the wiki repo: git@github.com:MatrixAI/js-polykey.wiki.git or git@github.com:MatrixAI/Polykey.wiki.git. This will be essential for putting images into the wiki so it doesn't need to link external resources or put images into the main repo.

    This repo might even be mirrored to our Gitlab as an explicit repo. I wonder how a README.md would be handled in the wiki repo.

  7. CMCDragonkai commented on Oct 25, 2021

    @CMCDragonkai
    CMCDragonkai
    on Oct 25, 2021
    MemberAuthor

    Remember no more gh-pages branch. Docs are going to come directly from master.

  8. CMCDragonkai commented on Oct 25, 2021

    @CMCDragonkai
    CMCDragonkai
    on Oct 25, 2021
    MemberAuthor

    While the branch is deleted, docs still need to be generated. So we should update this.

  9. CMCDragonkai commented on Dec 15, 2021

    @CMCDragonkai
    CMCDragonkai
    on Dec 15, 2021
    MemberAuthor

    Due to licensing inspired by Ethereum. We may want to do this:

  10. CMCDragonkai commented on Dec 15, 2021

    @CMCDragonkai
    CMCDragonkai
    on Dec 15, 2021
    MemberAuthor

    One problem... These are what has access to the macos runner.

    So we would need to rename https://gitlab.com/MatrixAI/open-source/js-polykey to https://gitlab.com/MatrixAI/open-source/Polykey-CLI. So that the github CLI benefits from the macos runner.

    This would mean that Polykey itself does not have macos runner. But that's ok for now, we can request it subsequently. Our tests currently only run on Nix anyway which is Linux only.

  11. CMCDragonkai commented on Dec 15, 2021

    @CMCDragonkai
    CMCDragonkai
    on Dec 15, 2021
    MemberAuthor

    The alternative is to stick with Polykey as GPL3, and then slowly move domains out to separate libraries when they can be relicensed.

  12. CMCDragonkai commented on May 11, 2022

    @CMCDragonkai
    CMCDragonkai
    on May 11, 2022
    MemberAuthor

    New plan:

    Polykey

    • js-polykey renames to Polykey
    • keeps all issues in js-polykey
    • license: GPL3 (possible move to APL3)
    • npm: polykey
    • not hosted on nix (it's not an application)
    • supports network API
    • embeddable as a library
    • bin/ code is extracted to Polykey-CLI

    Polykey-CLI

    • license: GPL3
    • npm: polykey-cli
    • nix: polykey-cli
    • extracts bin/ code out of js-polykey

    Polykey-Desktop - electron/tauri wrapper around library code

    • license: GPL3
    • not hosted on npm (it's not a JS library or application)
    • nix: polykey-desktop

    Polykey-Mobile - nativescript wrapper around library code

    • license: unknown
    • not hosted on npm (it's not a JS library or application)
    • not hosted on nix (it's not a desktop application)

    Updating on the issue description.

  13. CMCDragonkai
    added
    epicBig issue with multiple subissues
    on May 11, 2022
  14. CMCDragonkai
    changed the title [-]Merging js-polykey and Polykey repositories and fixing up names[/-] [+]Polykey naming and repository structure[/+] on May 11, 2022

  15. CMCDragonkai commented on May 14, 2022

    @CMCDragonkai
    CMCDragonkai
    on May 14, 2022
    MemberAuthor

    Because we don't control/own the wiki platform, instead of putting all of our docs there, our theory, guides, and tutorials will instead go directly to polykey.io/docs.

    Our reference can still be mostly generated through typedoc and hosted on github pages, we need some customisation there so we can link it back to polykey.io.

    Note that our libraries will only have reference documentation, they don't really need anything else, which is generated directly via typedoc.

    image

    Note that we may need a custom theme to be able to inject GTM into our generate docs page. More information should always be in the GH pages, since those links will not be nofollow. See: https://matrixai.github.io/TypeScript-Demo-Lib/ or https://matrixai.github.io/js-encryptedfs/. These links will all be done via github.io. These are all hosting the generated reference documentation.

  16. 3 remaining items

  17. CMCDragonkai commented on Jul 23, 2022

    @CMCDragonkai
    CMCDragonkai
    on Jul 23, 2022
    MemberAuthor

    However the wiki will be scheduled to go into the polykey.io/docs instead due to community/community#4992. GH wikis are not SEO friendly.

    However the wiki is still useful way of editing information compared to editing pages on our website. So a hybrid solution is to have the wiki available but then replicate/mirror the wiki to the website.

    How to do this? Well the wiki information is public, and we can use BE server to mirror the information. Because search engines cannot see the wiki anyway, it's fine to have the duplicate information.

    However polykey.io is hosted by webflow which has their own custom CMS. So how to connect to an external API (possibly serverless) backend to then load custom data. Most important it must do the rendering of HTML itself, and not just a mirror the HTML and present it via SSR, not fetched from JS.

  18. CMCDragonkai commented on Jul 23, 2022

    @CMCDragonkai
    CMCDragonkai
    on Jul 23, 2022
    MemberAuthor

    It looks like on webflow, it supports https://university.webflow.com/lesson/href-prefix which allows part of an existing website hosted by your own webserver to reverse proxy a particular url directory to webflow.

    However I'm looking to do the opposite, use webflow to reverse proxy a portion of the site to an alternative location. However they only allow this for enterprise: https://webflow.com/feature/reverse-proxy-service-for-enterprise

    So it looks like it will be time to to front webflow with our own reverse proxy to serve.

    There are 2 options here:

    1. Use cloudflare - it is already fronting polykey.io and we can use it's page rules and maybe other features to be able to route a subdirectory to a different backend server.
    2. Run our own system on AWS, and do the routing there, and upstream to webflow for the CMS and main site... etc. Hopefully the ALB is sufficient for this.

    Finally the underlying backend could work as a serverless platfrom like cloudflare workers, and then it just has to fetch the wiki markdown and render the correct page.

    This does limit the wiki's formatting to what the HTML is shown. See: https://stackoverflow.com/questions/38480701/can-a-github-wiki-embed-html and https://github.com/bryanbraun/github-wiki-html-test/wiki. Note that even if github wiki itself removes those tags, the code is still in the wiki git, and when we mirror we can render that with custom CSS as well if needed. We will need to incorporate this https://webmasters.stackexchange.com/a/76927

    Seems like something to be automated with pulumi when we get to it.

  19. CMCDragonkai
    transferred this issue fromon Jul 23, 2022

  20. CMCDragonkai commented on Jul 23, 2022

    @CMCDragonkai
    CMCDragonkai
    on Jul 23, 2022
    MemberAuthor

    All issues and wiki and asset files transferred over. Just need to move over a README.md.

  21. CMCDragonkai commented on Jul 23, 2022

    @CMCDragonkai
    CMCDragonkai
    on Jul 23, 2022
    MemberAuthor

    Old Polykey repository can be deleted now. Going to first archive it, and then delete it later.

  22. CMCDragonkai commented on Jul 23, 2022

    @CMCDragonkai
    CMCDragonkai
    on Jul 23, 2022
    MemberAuthor

    Now need to update gitlab.

  23. CMCDragonkai commented on Jul 23, 2022

    @CMCDragonkai
    CMCDragonkai
    on Jul 23, 2022
    MemberAuthor

    Zenhub and gitlab swapped over. Seems to be working fine.

  24. CMCDragonkai commented on Jul 23, 2022

    @CMCDragonkai
    CMCDragonkai
    on Jul 23, 2022
    MemberAuthor

    Still need to do this: https://docs.npmjs.com/deprecating-and-undeprecating-packages-or-package-versions

    But to do it well, we should be publishing the polykey package @tegefaulkes @emmacasolin

  25. CMCDragonkai commented on Jul 23, 2022

    @CMCDragonkai
    CMCDragonkai
    on Jul 23, 2022
    MemberAuthor

    @emmacasolin @tegefaulkes make sure to swap the remotes for origin and cicd to point to:

    [nix-shell:~/Projects/Polykey]$ git remote -v
cicd    git@gitlab.com:MatrixAI/open-source/Polykey.git (fetch)
cicd    git@gitlab.com:MatrixAI/open-source/Polykey.git (push)
origin  git@github.com:MatrixAI/Polykey.git (fetch)
origin  git@github.com:MatrixAI/Polykey.git (push)

    Use git remote set-url origin git@github.com:MatrixAI/Polykey.git.

    Rename the directory js-polykey to Polykey and then just checkout staging and git pull. That will be sufficient to update anything you're using.

  26. CMCDragonkai commented on Jul 23, 2022

    @CMCDragonkai
    CMCDragonkai
    on Jul 23, 2022
    MemberAuthor

    Codesee now integrated, so we should get maps for each PRs now.

  27. CMCDragonkai commented on Jul 23, 2022

    @CMCDragonkai
    CMCDragonkai
    on Jul 23, 2022
    MemberAuthor

    We should aim to get all staging tests passing (or disabling whatever) in order to release a new version, so this issue can be closed.

  28. CMCDragonkai commented on Jul 24, 2022

    @CMCDragonkai
    CMCDragonkai
    on Jul 24, 2022
    MemberAuthor

    Going to split out this issue to subissues, as the main job is done.

  29. CMCDragonkai commented on Jul 24, 2022

    @CMCDragonkai
    CMCDragonkai
    on Jul 24, 2022
    MemberAuthor

    Publishing tasks moved to #268.

  30. CMCDragonkai commented on Jul 24, 2022

    @CMCDragonkai
    CMCDragonkai
    on Jul 24, 2022
    MemberAuthor

    Wiki publishing going to MatrixAI/Polykey-Docs#1.

  31. CMCDragonkai
    closed this as completedon Jul 24, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

documentationImprovements or additions to documentationepicBig issue with multiple subissuesprocedureAction that must be executed

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions