Mobile shdwDrive: Solana Crossroads 2024 Recap
The GenesysGo team debuted a Proof of Concept platform that represented two years worth of learning and engineering
Audience members with an Android device were provided with a QR code and asked to download the newly minted shdwDrive app from the Google Play store. After installing the app, audience members were able to watch, in real time, as their Android phones all began to trustlessly communicate with one another using shdwDrive's uniquely designed peer2peer networking protocol and pool together the unused hard drive space on each phone into a cohesive storage pool.
Those audience members with Android devices were brought together and became the world's first ever decentralized storage network that was being entirely powered by mobile devices. This means, all the operations necessary to run shdwDrive (data ingestion, encryption, consensus, member management, and more) were taking place exclusively on those mobile devices in that room and that room only.
These newly minted mobile shdwOperators were then able to watch, once again in real time, as the unused hard drive space their phones had contributed began to fill up with data. This mini shdwDrive v2 network was directly ingesting Solana's ledger history directly from the Solana Validator network with no middleware servers in between. This resulted in a second "first time ever" event, where the actual individual members of the Solana ecosystem were playing an active and firsthand role in protecting and being the custodians of Solana's ledger history.
Let's talk more, on a high level, about why this Proof of Concept deployment is significant and what it means for the upcoming release of shdwDrive v2 later on this year.
The World's First and Only Decentralized Storage Network, Powered Exclusively on Mobile Devices
On Friday, May 10th 2024, every person in the presentation hall of the Solana Crossroads event was a part of web3 history. The shdwDrive mobile app served as a Proof of Concept demonstrating that the thousands of petabytes of unused hard drive space sitting on the mobile phones of billions of individuals could be monetized for their benefit.
The Android platform is far and away the most popular mobile phone platform in the world, with a little over 75% of the world’s mobile devices running Android. Android devices are especially prevalent in developing countries where the amount of Android users can reach a staggering 85-90%. All together, there are approximately 3.8 billion mobile phones around the world being powered by the Android platform. All of them with unused hard drive space that is sitting idle and completely untapped.
shdwDrive is the first decentralized data storage platform that enables users to turn their phones into appreciating assets by monetizing their unused hard drive space in a way that is completely trustless and, on the network level, fully censorship resistant. Each phone acts as both a storage node and a gateway node to ensure both data ingestion and data storage are a fully decentralized process that is governed exclusively by network consensus.
Ingesting and Storing Solana Ledger History live at Solana Crossroads!
At the heart of shdwDrive is a sophisticated blend of cutting-edge cryptographic techniques and innovative networking protocols, designed to ensure secure, private, and efficient decentralized storage. The most remarkable aspect is that all these processes are executed entirely on Android mobile phones, making this a groundbreaking achievement in decentralized technology.
During GenesysGo’s presentation at Solana Crossroads, the audience witnessed a live demonstration of shdwDrive’s capabilities. Android phones in the audience ingested data directly from the Solana Validator network via the Geyser plugin. The Geyser plug-in was designed by the Solana Labs team to ensure that data could be received from the Solana Validators in a way that ensures the data is current and accurate without requiring any middleware. The shdwDrive app enabled the phones to ingest Solana’s Ledger history directly from the Solana Validator network in real-time, ensuring that anyone accessing Solana’s Ledger history on shdwDrive knows with full certainty that it has not been altered or passed through the hands of a middleware service. The data was then distributed across the network as erasure-coded shreds through shdwDrive's network consensus mechanism. This process ensured that even if some phones went offline or data fragments were lost, the original data could still be reconstructed from the remaining fragments, ensuring high availability and fault tolerance.
Direct Trustless Data Transmission: One of the major achievements of this Proof of Concept is that the Solana ledger history data is transmitted directly from the Solana Validator network to the shdwDrive network without passing through any middleware or third-party providers. This direct transmission ensures that users can rely on the data's integrity, knowing it has not been tampered with during transit. The data moves seamlessly from one trustless network to another, maintaining the highest level of security and trust.
Erasure Coding: To safeguard data and ensure its durability, shdwDrive uses erasure coding. This process breaks down arbitrary file bytes into smaller fragments, which are then distributed across multiple storage nodes—all of which are Android mobile phones. Even if some phones go offline or data fragments are lost, the original data can still be reconstructed from the remaining fragments, ensuring high availability and fault tolerance.
Zero-Knowledge Proofs: To maintain the integrity and confidentiality of the stored data, shdwDrive employs zero-knowledge proofs (zk-proofs). These cryptographic proofs enable the verification of data and operations without revealing the data itself. Remarkably, all zk-proof operations are conducted on the mobile phones, proving that high-level cryptographic computations can be efficiently managed by everyday devices.
Bulletproofs and Pedersen Commitments: Within the realm of zk-proofs, shdwDrive leverages bulletproofs and Pedersen commitments to enhance security and privacy:
- Pedersen Commitments: These are used to securely commit to data values on mobile devices. A Pedersen commitment allows a user to commit to a value while keeping it hidden and ensuring the value cannot be altered once committed. This binding and hiding property is crucial for maintaining data integrity and confidentiality within the network, all managed by the Android phones.
- Bulletproofs: To verify data operations efficiently and securely, shdwDrive uses bulletproofs. These zero-knowledge proofs are both short and efficient, enabling quick verification without the need for a trusted setup. Bulletproofs are particularly useful for implementing range proofs, ensuring that committed data values fall within acceptable limits without revealing the actual values. Astonishingly, these proofs are generated and verified by the mobile phones themselves.
Proof Verification and Challenges: Once the zk-proofs are generated on the mobile phones, they undergo independent verification through a series of challenges, also executed on the phones. This process ensures that all proofs are valid and that the data operations comply with the network’s protocols. By using these advanced cryptographic techniques, shdwDrive maintains a trustless and censorship-resistant storage environment.
Unmatched Data Integrity: shdwDrive is the first data storage network where it is impossible for a bad actor to introduce corrupted data into the network. This is achieved through the combination of erasure coding and leveraging two kinds of specific zk-proofs, bulletproofs and Pedersen commitments. These technologies ensure that any attempt to alter or corrupt data is detectable and can be prevented by the network’s consensus mechanism.
Trustless Data Existence: The integration of these cryptographic methods means that the existence and integrity of your data are completely trustless. Users do not need to rely on any central authority or intermediaries to ensure their data is stored securely and remains unaltered. There is also no need for any kind of intermediary data layer that scrapes to data randomly to verify authenticity (although, any data availability layer is more than welcome to audit as they see fit) as it is mathematically impossible for bad data to be served. Every operation within the shdwDrive network is validated by network consensus, providing unparalleled assurance of data security and privacy.
The ability to perform such complex cryptographic operations entirely on Android mobile phones is the true innovation that makes the shdwDrive network incredibly unique relative to other decentralized storage platforms.
Data Reconstruction and Network Repair
shdwDrive’s use of advanced cryptographic techniques, particularly erasure coding, ensures robust data integrity and availability. Erasure coding breaks down files into multiple smaller fragments, which are then distributed across the network of Android mobile devices. In the event of data loss, shdwDrive can efficiently reconstruct the original files from the remaining fragments. The shdwDrive network is so resilient that even if 67% of the data were lost due to device failures or network issues, the remaining 33% of data fragments would still be sufficient to fully restore the lost data. This ensures that users’ data remains intact and accessible, providing a high level of fault tolerance and reliability. This resiliency number could technically go as high as 80% but anything about 67% begins to suffer dramatically from diminished returns and ultimately just ends up wasting network resources. By leveraging these techniques, shdwDrive not only maintains data integrity but also facilitates seamless network repairs, guaranteeing continuous and secure access to stored information.
The Benefits of a Mobile Powered Storage Network
The mobile architecture of shdwDrive is a groundbreaking innovation that ensures its performance scales seamlessly with the addition of more mobile phones. Each phone in the network acts as both a storage node and a gateway node for data ingestion and retrieval requests. This means that as more mobile devices join the network, the collective capability to ingest, process, and store data increases exponentially. The decentralized nature of this architecture eliminates bottlenecks and single points of failure, making the network more robust and resilient.
A mobile-only network has the potential to out-scale traditional server-based networks due to the sheer number of mobile devices available worldwide. With over 3.8 billion Android devices globally, and a significant portion of these in developing countries where Android usage is prevalent, the available storage capacity is immense. Unlike traditional server farms, which require significant investment in infrastructure and maintenance, a mobile-powered network leverages existing devices, dramatically reducing costs and environmental impact. As more users join shdwDrive, contributing their unused hard drive space, the network's capacity and performance grow, surpassing the scalability limits of conventional data centers.
Moreover, shdwDrive stands out as the most climate-friendly decentralized storage network in existence. Traditional server farms have a massive carbon footprint and consume millions of gallons of groundwater to stay functional, contributing to environmental degradation. In contrast, shdwDrive harnesses the existing resources of mobile devices, significantly reducing energy consumption and the overall ecological footprint. By leveraging the vast number of Android devices globally, shdwDrive transforms unused mobile storage into a cohesive, high-performance storage network, all while minimizing environmental impact.
This coordination between mobile phones ensures efficient data distribution and retrieval, enhancing the overall performance and scalability of the platform. As a result, shdwDrive not only democratizes data storage but also sets a new standard for decentralized storage solutions powered entirely by mobile devices.
Mobile Only Architecture Results in Superior Security for shdwDrive v2
One of the standout features of shdwDrive is its ability to tap into the robust security infrastructure inherent to all Android devices. Android’s security model is designed to protect users and their data through multiple layers of defense. Each app operates within its own sandbox, preventing malicious applications from accessing sensitive data or affecting other apps. The Verified Boot process ensures the integrity of the device’s software, while SELinux enforces strict access controls to limit the impact of potential vulnerabilities. Full-disk encryption safeguards user data, even if the device is lost or stolen, and Google Play Protect continuously scans for malicious behavior. Additionally, Android’s granular permissions model gives users control over app access to sensitive information. These safeguards have been built in to alert the device if the owner or another application is attempting to alter the device or access data that should not be accessed. This ensures that all data stored on the mobile devices powering the shdwDrive v2 network is incredibly secure, in fact said data is far more secure on a mobile powered network than it would be using traditional Linux powered server infrastructure.
Moreover, the mobile-only architecture of shdwDrive allows the GenesysGo team to build on a platform with fewer attack surfaces compared to traditional Linux server architecture. This significantly reduces the complexity of securing the network, making the development of shdwDrive v2 a faster and more resource-efficient exercise. By leveraging the advanced security features of Android devices and minimizing potential vulnerabilities, shdwDrive ensures that users can trust the safety and integrity of their stored data while also benefiting from a more streamlined and efficient development process.
What does the future hold for shdwDrive?
The successful Proof of Concept deployment of shdwDrive v2’s mobile powered architecture at Solana Crossroads 2024 marks a pivotal moment in the evolution of decentralized storage. By leveraging the ubiquitous presence and untapped potential of Android mobile devices, shdwDrive not only provides a scalable and resilient storage solution but also sets a new standard for environmental sustainability in the blockchain space. The ability to perform complex cryptographic operations entirely on mobile phones, coupled with the robust data integrity mechanisms, ensures that shdwDrive is both innovative and secure. As we look forward to the upcoming release of shdwDrive v2, this Proof of Concept serves as a powerful testament to the transformative potential of mobile-powered decentralized storage networks. With shdwDrive, we are not just redefining data storage; we are democratizing it, making it more accessible, reliable, and environmentally friendly for users around the globe.
Over the remainder of 2024, the GenesysGo team’s full attention will be focused solely on the deployment of shdwDrive v2. The unveiling of the last piece of the puzzle that answers questions like, “How can shdwDrive provide inexpensive storage and still remain a viable platform?” are answered now that mobile powered shdwDrive has been unveiled. Additionally, this last piece of the puzzle lays the groundwork for a fast and seamless deployment of shdwDrive’s compute services and what will ultimately become the world’s first truly decentralized DNS platform that bypasses traditional internet infrastructure all together.
