trusted nodes (WebOfTrust) / shared storage with friends / coalescing multiple clients #397

The idea is to add a friend list and ACL features (like private IPFSs) which allows multi-device access on a private storage redundantly sharded between friends. Additionally, files can be shared with a specific group (like 1, 5, 10, all friend(s)).

This makes the transition between private, peer-group and public data for the user much easier and allows also some additional features and benefits for the network.

This feature requires some kind of direct notification messages, e.g. notify other nodes about new files they need to store.

Full list of ideas:

• bootstrapping can primarily connect to friends
  • avoiding unnecessary ipfs-network load on startup
  • querying friend's nodes first when searching for data
  • lower amount of routing necessary
  • lower delay
  • allows relay for NAT traversal for friends

• a friend list in IPFS would show up in all apps which connect to the API

• the friend list can be used as ACL for data on a node
  • ability to share a file just with the friends without encryption or a separate private ipfs-network

• ability to send notifications to all (a group of) friends
  • allows e.g. for RSS like, pushed updates of new pinned peer-group/public files
  • notify friend(s) of a file which is explicitly shared (easy file transfer notification)
  • apps could send custom messages with this feature (e.g. for chats, social networks, calls)

• multi-device ACL: coalesce multiple nodes-ids to one 'friend' entry
  • second device can be added by signing with the first private key a message
  • device can be removed by signing a remove message with a different device's private key
  • select a user gives all devices of the user access
  • address multiple nodes together e.g. notifications
  • buffered notifications: one device of a user can hold new notifications, while another device is offline

• storage sharing with friends (quota per friend; quota for all friends)
  • friendship requests needs to be a push+pin to connect feature, to avoid spamming
  • when you acknowledged a friendship request, you're asked how much storage you want to provide
  • friends can store data for a user with sharding/redundancy
  • simple scrambling of data: placing just every n-bit of original data in a chunk per friend
  • diff updates are possible (replace of some chunks)
  • no 3rd party software needed for encryption (portability)
  • no encryption layer: lower CPU requirements
  • compression of the datastore is still beneficial
  • identical files can still be identified (by the node who stores them on friend's nodes)
  • multi-device ACL allows different devices to access the same data (without exchanging a crypto key)
  • disaster recovery: all friends can manually accept a new device for a user to give him access to his data back

• remote data integrity checks (proofs that a friend still has data you sent him)
  • background task: asking for data which wasn't accessed for a while to confirm redundancy
  • friend's node is asked to XOR a salt to the file and send the checksum
  • salted-checksum-list can be calculated before seeding, avoiding that the data has to remain 'local'

• basically building a Web Of Trust
  • better resilience to DDoS attacks (e.g. limit new connections if they are excessively established)
  • creates a grid of geographically local nodes, reducing the overall latency of the network
  • a friend can introduce a friend (cryptographically signed) to a user
  • we might be able to skip some cryptographically checks or DoS filter on network traffic received from friends
  • avoid searches in the network: link shares can be searched in the cache of the friends first
  • friends can use special network features, like relays
  • the maximal processing time/bandwidth can be higher for friends, to allow for a more stable network
  • peer-group caching of 'hot' files (with a lot of accesses)
    • a node can ask the friend's nodes to store chunks of a file in their cache
    • automatically increases the availably of files which are accessed often
    • avoids that one node has to serve many clients which don't reupload the file (fast)
    • will cascade if there's still a lot of access on the file, since the friends can ask their friends to hold the data
  • user-initiated peer-group caching
    • if selected a new file will be spread to the friend's cache (if there's some space left)
    • user can specify a hold time (info for the garbage collector additionally to requests on it)
    • allows the user to upload a file to the network if he has limited bandwidth/time
    • hot chunks of the file will be cached further in the network (peer-group caching of 'hot' files)