Tech Tuesday: The Bit-World We Live In
25 July 2017 |
Roland Becsi | About a 3 minute read
The world’s most valuable resource is no longer oil, but data. Companies like Google, Facebook and Amazon trade with data, they sell your browsing history, search habits and personal information so you can get customised ads, offers and so on.
This data either lives in the cloud, on powerful servers owned by these companies, or in small, dusty boxes thrown in the corner of a basement. Who knows for sure – we trust them with our valuable information nonetheless. In some cases the transfer or migration might cause losses, outages can cause server downtimes, and if we need to store and access our data we want to be sure it’s safe and always accessible.
Secure storage is one area of concern, in addition there is also the challenge around bandwidth. Traffic is sent through HTTP connection whose protocol uses a single thread to download a file. This may cause bandwidth issues in the case of larger files. Bittorrent uses a peer-to-peer file sharing system which is a more efficient way of distributing file fragments and increasing download speed although it is vulnerable to some exploitative bandwidth sharing strategies.
Are there any new initiatives?
Interplanetary File System (IPFS) is a peer-to-peer hypermedia protocol that aims to make the web faster, safer, and more open. Here, every user is a peer on the network, and the uploaded files are shared, signed with a cryptographic hash and then distributed across the network.
The data will not be owned by anyone, is available at any given time, and can be downloaded faster. In some scenarios the P2P video delivery could save 60% in bandwidth costs. IPFS makes it possible to distribute high volumes of data with high efficiency. Zero duplication also means savings in storage.
Although this technology is relatively new, there are already many enterprise use cases, for example https://storj.io, a blockchain based IPFS storage provider.
Storj is a file sharing service where any user can be a peer. The company itself doesn’t store or own any of the uploaded data. Everyday users can sign up for a fee to store their large files and some of them sign up to store the data for revenue.
If you’re interested renting out the unused space on your hard drive, you can sign up here and earn small fees after the usage https://storj.io/share.html.
How does AND Digital use IPFS?
Currently one of our new clients in the Medical industry, utilises the IPFS technology to store large medical files. The blockchain technology is good to store small, text based data but large files may slow down the entire network. Instead, we store the files in IPFS and only put the hash reference to the distributed ledger. This is a very efficient way to store any kind of information securely, share it with whoever we want, and minimise data loss upon transfer.
By architecture, every GP has its own server, which holds the ledger and an IPFS server. The files being uploaded to the GP node and get pinned, meaning that the file is stored as a whole, also being sharded and distributed across the client’s network. This way patients are accessing the whole file for best performance but if the GP node is down, they can still reconstruct the file from the shards.
Look out for the follow up article to this, where we will set up a local IPFS network to read and write files to it.Read More From This Author
Senior Product Developer (London)
Champion software quality and technical vision for AND and our clients, work on large-scale projects and help junior and mid developers grow in their roles.
Technologies you will be using
Next People Strategy & Operations – Senior Consultant (London)
Lead in building our strategy and the operationalisation of our approach to establishing new clubs across the UK, Europe and beyond!I'm Interested
Squad Lead (Leeds)
Lead and inspire a squad to succeed, bringing your passion for managing and developing people through career coaching and mentoring.I'm Interested