Examples: Add repository uploader

examples/README.md

@@ -1,5 +1,6 @@
 # Usage examples
 
 * [repository](repository)
-* [client](client_example)
-* [repository built with low-level Metadata API](manual_repo)
+* [client](client)
+* [uploader tool](uploader)
+* [Low-level Metadata API examples](manual_repo)

examples/repository/README.md

@@ -9,7 +9,11 @@ This TUF Repository Application Example has the following features:
 - Serves metadata and targets on localhost (default port 8001)
 - Simulates a live repository by automatically adding a new target
   file every 10 seconds.
-
+- Exposes a small API for the [uploader tool example](../uploader/). API POST endpoints are:
+  - `/api/role/<ROLE>`: For uploading new delegated targets metadata. Payload
+    is new version of ROLEs metadata
+  - `/api/delegation/<ROLE>`: For modifying or creating a delegation for ROLE.
+    Payload is a dict with one keyid:Key pair
 
 ### Usage
 
@@ -18,4 +22,4 @@ This TUF Repository Application Example has the following features:
 ```
 Your repository is now running and is accessible on localhost, See e.g.
 http://127.0.0.1:8001/metadata/1.root.json. The
-[client example](../client_example/README.md) uses this address by default.
+[client example](../client/README.md) uses this address by default.

examples/repository/_simplerepo.py

@@ -4,15 +4,19 @@
 """Simple example of using the repository library to build a repository"""
 
 import copy
+import json
 import logging
 from collections import defaultdict
 from datetime import datetime, timedelta
 from typing import Dict, List
 
 from securesystemslib import keys
-from securesystemslib.signer import Signer, SSlibKey, SSlibSigner
+from securesystemslib.signer import Key, Signer, SSlibKey, SSlibSigner
 
+from tuf.api.exceptions import RepositoryError
 from tuf.api.metadata import (
+    DelegatedRole,
+    Delegations,
     Metadata,
     MetaFile,
     Root,
@@ -116,7 +120,7 @@ def close(self, role: str, md: Metadata) -> None:
             self._targets_infos[f"{role}.json"].version = md.signed.version
 
     def add_target(self, path: str, content: str) -> None:
-        """Add a target to repository"""
+        """Add a target to top-level targets metadata"""
         data = bytes(content, "utf-8")
 
         # add content to cache for serving to clients
@@ -131,3 +135,73 @@ def add_target(self, path: str, content: str) -> None:
         # update snapshot, timestamp
         self.snapshot()
         self.timestamp()
+
+    def submit_delegation(self, rolename: str, data: bytes) -> bool:
+        """Add a delegation to a (offline signed) delegated targets metadata"""
+        try:
+            logger.debug("Processing new delegation to role %s", rolename)
+            keyid, keydict = next(iter(json.loads(data).items()))
+            key = Key.from_dict(keyid, keydict)
+
+            # add delegation and key
+            role = DelegatedRole(rolename, [], 1, True, [f"{rolename}/*"])
+            with self.edit("targets") as targets:
+                if targets.delegations is None:
+                    targets.delegations = Delegations({}, {})
+
+                targets.delegations.roles[rolename] = role
+                targets.add_key(key, rolename)
+
+        except (RepositoryError, json.JSONDecodeError) as e:
+            logger.info("Failed to add delegation for %s: %s", rolename, e)
+            return False
+
+        logger.debug("Targets v%d", targets.version)
+
+        # update snapshot, timestamp
+        self.snapshot()
+        self.timestamp()
+
+        return True
+
+    def submit_role(self, role: str, data: bytes) -> bool:
+        """Add a new version of a delegated roles metadata"""
+        try:
+            logger.debug("Processing new version for role %s", role)
+            if role in ["root", "snapshot", "timestamp", "targets"]:
+                raise ValueError("Only delegated targets are accepted")
+
+            md = Metadata.from_bytes(data)
+            for targetpath in md.signed.targets:
+                if not targetpath.startswith(f"{role}/"):
+                    raise ValueError(f"targets allowed under {role}/ only")
+
+            targets_md = self.role_cache["targets"][-1]
+            targets_md.verify_delegate(role, md)
+            if role in self.role_cache:
+                current_md = self.role_cache[role][-1]
+                current_ver = current_md.signed.version
+            else:
+                current_ver = 0
+
+            if md.signed.version != current_ver + 1:
+                raise ValueError("Invalid version {md.signed.version}")
+
+        except (RepositoryError, ValueError) as e:
+            logger.info("Failed to add new version for %s: %s", role, e)
+            return False
+
+        # Checks passed: Add new delegated role version
+        self.role_cache[role].append(md)
+        self._targets_infos[f"{role}.json"].version = md.signed.version
+        logger.debug("%s v%d", role, md.signed.version)
+
+        # To keep it simple, target content is generated from targetpath
+        for targetpath in md.signed.targets:
+            self.target_cache[targetpath] = bytes(f"{targetpath}", "utf-8")
+
+        # update snapshot, timestamp
+        self.snapshot()
+        self.timestamp()
+
+        return True

examples/repository/repo

@@ -13,25 +13,61 @@ import argparse
 import logging
 import sys
 from datetime import datetime
-from http.server import BaseHTTPRequestHandler, HTTPServer
+from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
 from time import time
 from typing import Dict, List
 
 from _simplerepo import SimpleRepository
 
+from tuf.api.serialization.json import JSONSerializer
+
 logger = logging.getLogger(__name__)
 
 
 class ReqHandler(BaseHTTPRequestHandler):
-    """HTTP handler to serve metadata and targets from a SimpleRepository"""
+    """HTTP handler for the repository example application
+
+    Serves metadata, targets and a small upload API using a SimpleRepository
+    """
+
+    def do_POST(self):
+        """Handle POST requests, aka the 'uploader API'"""
+
+        content_len = int(self.headers.get("content-length", 0))
+        data = self.rfile.read(content_len)
+
+        if self.path.startswith("/api/delegation/"):
+            role = self.path[len("/api/delegation/") :]
+            if not self.server.repo.submit_delegation(role, data):
+                return self.send_error(400, f"Failed to delegate to {role}")
+        elif self.path.startswith("/api/role/"):
+            role = self.path[len("/api/role/") :]
+            if not self.server.repo.submit_role(role, data):
+                return self.send_error(400, f"Failed to submit role {role}")
+        else:
+            return self.send_error(404)
+
+        self.send_response(200)
+        self.end_headers()
 
     def do_GET(self):
+        """Handle GET: metadata and target files"""
+        data = None
+
         if self.path.startswith("/metadata/") and self.path.endswith(".json"):
-            self.get_metadata(self.path[len("/metadata/") : -len(".json")])
+            data = self.get_metadata(
+                self.path[len("/metadata/") : -len(".json")]
+            )
         elif self.path.startswith("/targets/"):
-            self.get_target(self.path[len("/targets/") :])
+            data = self.get_target(self.path[len("/targets/") :])
+
+        if data is None:
+            self.send_error(404)
         else:
-            self.send_error(404, "Only serving /metadata/*.json")
+            self.send_response(200)
+            self.send_header("Content-length", len(data))
+            self.end_headers()
+            self.wfile.write(data)
 
     def get_metadata(self, ver_and_role: str):
         repo = self.server.repo
@@ -44,35 +80,28 @@ class ReqHandler(BaseHTTPRequestHandler):
             ver = int(ver_str)
 
         if role not in repo.role_cache or ver > len(repo.role_cache[role]):
-            self.send_error(404, f"Role {role} version {ver} not found")
-            return
+            return None
 
-        # send the metadata json
-        data = repo.role_cache[role][ver - 1].to_bytes()
-        self.send_response(200)
-        self.send_header("Content-length", len(data))
-        self.end_headers()
-        self.wfile.write(data)
+        # return metadata
+        return repo.role_cache[role][ver - 1].to_bytes(JSONSerializer())
 
     def get_target(self, targetpath: str):
-        repo: SimpleRepository = self.server.repo
-        _hash, _, target = targetpath.partition(".")
+        repo = self.server.repo
 
-        if target not in repo.target_cache:
-            self.send_error(404, f"target {targetpath} not found")
-            return
+        # unimplement the dumb hashing scheme
+        # TODO: maybe use hashed paths as the target_cache key
+        dir, sep, hashname = targetpath.rpartition("/")
+        _, _, name = hashname.partition(".")
+        target = f"{dir}{sep}{name}"
 
-        # TODO: check that hash actually matches -- or use hash.targetpath as target_cache keys?
+        if target not in repo.target_cache:
+            return None
 
         # send the target content
-        data = repo.target_cache[target]
-        self.send_response(200)
-        self.send_header("Content-length", len(data))
-        self.end_headers()
-        self.wfile.write(data)
+        return repo.target_cache[target]
 
 
-class RepositoryServer(HTTPServer):
+class RepositoryServer(ThreadingHTTPServer):
     def __init__(self, port: int):
         super().__init__(("127.0.0.1", port), ReqHandler)
         self.timeout = 1

examples/uploader/README.md

@@ -0,0 +1,44 @@
+# TUF Uploader Tool Example
+
+:warning: This example uses the repository module which is not considered
+part of the python-tuf stable API quite yet.
+
+This is an example maintainer tool: It makes it possible to add delegations to
+a remote repository, and then to upload delegated metadata to the repository.
+
+Features:
+   - Initialization (much like the [client example](../client/))
+   - Claim delegation: this uses "unsafe repository API" in the sense that the
+     uploader sends repository unsigned data. This operation can be
+     compared to claiming a project name on PyPI.org
+   - Add targetfile: Here uploader uses signing keys that were added to the
+     delegation in the previous step to create a new version of the delegated
+     metadata. The repository will verify signatures on this metadata.
+
+The used TUF repository can be set with `--url` (default repository is
+"http://127.0.0.1:8001" which is also the default for the repository example).
+In practice the uploader tool is only useful with the repository example.
+
+### Usage with the repository example
+
+In one terminal, run the [repository example](../repository/) and leave it running:
+```console
+examples/repository/repo
+```
+
+In another terminal, run uploader:
+
+```console
+# Initialize with Trust-On-First-Use
+./uploader tofu
+
+# Then claim a delegation for yourself (this also creates a new signing key):
+./uploader add-delegation myrole
+
+# Then add a new downloadable target file to your delegated role (to keep the
+# example simple, the target file content is always the targetpath):
+./uploader add-target myrole myrole/mytargetfile
+```
+
+At this point "myrole/mytargetfile" is downloadable from the repository
+with the [client example](../client/).