Sapere Aude
Tag: distributed
contra “innovation only happens in person” beliefs:
- it’s not actually that hard to collaborate productively at a distance in academia, at least once you’ve gotten to know someone.
- innovation requires ever more collaboration among specialists as knowledge accumulates.
- Over time, falling travel and communication costs have increasingly favored building those teams by turning to remote colleagues with the right specialization.
File systems unfit as distributed storage backends
Breaking the assumption that a distributed storage backend should clearly be layered on top of a local file system allowed Ceph to introduce a new storage backend called BlueStore with much better performance and predictability, and the ability to support the changing storage hardware landscape.
Fast key-value stores, an idea whose time has come and gone
In ProtoCache, replacing the RInK with stateful application servers resulted in a 29-57% median latency improvement and 40%+ reduction in CPU.
While Tupperware shares many common features with other cluster management systems, such as Kubernetes and Mesos, it distinguishes itself in the following areas: Seamless support for stateful services. A single control plane managing servers across data centers to help automate intent-based container deployment, cluster decommission, and maintenance. Transparent sharding of the control plane to scale out. An elastic compute approach to shift capacity among services in real time.
Efficient, reliable cluster management at scale with Tupperware
What is the family of problems that can be consistently computed in a distributed fashion without coordination, and what problems lie outside that family?
How do you know how well your large kubernetes cluster is performing? Is a particular change worth deploying? Can you quantify the ROI? To do that, you’re going to need some WSC-wide metric of performance. Not so easy! The WSC may be running 1000s of distinct jobs all sharing the same underlying resources. Developing a load-testing benchmark workload to accurately model this is ‘practically impossible.’ Therefore, we need a method that lets us evaluate performance in a live production environment. Google’s answer is the Warehouse Scale performance Meter (WSMeter), “a methodology to efficiently and accurately evaluate a WSC’s performance using a live production environment.” At WSC scale, even small improvements can translate into considerable cost reductions. WSMeter’s low-risk, low-cost approach encourages more aggressive evaluation of potential new features.
The talking heads will soon turn their attention to this. Will we get thorium reactors, one per block? Vertical farming?
The resilient community has broad applicability beyond just improving the ability of those of us in developed economies to preserve wealth and a quality of life despite severe system shocks. It can also be applied to the problems of counter-insurgency in semi-modern urban environment (to radically update a process that was built for the last century) and provide the potential for organic development in underdeveloped areas of the world.
one of the most data-intensive apps in the world today. from 1 PB / s to a mere 1 GB / s.
using open source warfare techniques. this will make it impossible for washington to co-opt them. interesting times.
There are over 2B literate, mobile phone subscribers in the developing world, many living on less than $5 a day. Corporations pay people to accomplish millions of simple text-based tasks. txteagle enables these tasks to be completed via text message by ordinary people around the globe.
between harnessing slacker energy and txteagle, we have not even scratched the surface of outsourcing. if you are an anti-outsourcing douchetard, it is time to dig in your heels. you won’t like the future.
Gregor J. Rothfuss 