it is very clear that the response to sandy would have massively improved if people had a way of coordinating, instead of all the stories of volunteers being turned away, collected yoga mats going unused, etc. rather than reinventing wheels poorly, hopefully the city will look into sahana for their next hackathon
Tag: software
Synthesis optimization

The software combines long syntheses of compounds into shorter and more economical routes, and identifies suspicious chemical recipes that could lead to chemical weapons.
Their main trick seems to be to combine multiple steps into steps that can happen at once, what they call “one pot”.
2022-02-23: Nice analogy on the energy landscape for reactions, twisty pathways along the edges of steep ridges.
What would it take to have software that showed you the best synthesis for a given compound, though? Now that’s a dream that even I think is out of reach for us, at least for as far out into the future as I can imagine. And this paper illustrates why! Look at all the tiny variations that end up making a difference, and sometimes a big difference. If we could model or compute our way to the answers in such situations, believe me, we would do that rather than set up endless arrays of reactions just to see what happens. Everyone who’s done research-level synthetic organic chemistry has experienced this: you flip one chiral center in your molecule, or made a chain 1 carbon longer, you change the solvent from one ether to another, raise or lower the temperature a bit, switch a sodium salt for a potassium one, change a ligand on your palladium catalyst, whatever, and all sorts of craziness breaks loose. And it’s often not easy to see why things changed so much. Ex post facto you can sometimes come up with hypotheses, and use those to fix things up if you’re right. But there are plenty of throw-your-hands-up moments that just never get explained at all.
Organic chemistry wobbles and teeters across an energy landscape that (from the viewpoint of any given reaction) is full of huge hills, deep valleys, and twisty little pathways that are followed by walking along the edges of steep, crumbling ridges. But from a distance, all that topography is compressed into a pretty narrow thermodynamic range. The differences between a reaction working and not working, between it giving you mostly Product A, mostly Product B, mostly returned starting material, or mostly scorched pan drippings are energetically very small. All sorts of little changes can send things off in different directions, and these can be largely inside the error bars of our attempts to model them. Organic chemistry is indeed a mature science, but don’t confuse that with thinking that it’s a solved problem. If you just want the molecules, damn the cost, to answer other questions we can generally provide them. But if you want them made elegantly, you’ll need to take a seat – and you better have packed some lunch with you.
Bullshit jobs
The paradox is this. A job seeker is looking for a well-defined job. But the trend seems to be that if a job can be defined, it can be automated or outsourced. The marginal product of people who need well-defined jobs is declining. The marginal product of people who can thrive in less structured environments is increasing.
See also zero marginal product workers what fraction of the economy is pretend work, appearing to be busy?
2013-08-20: On the predicament of ZMP (0 marginal product) workers:
Huge swathes of people spend their entire working lives performing tasks they secretly believe do not really need to be performed. The moral and spiritual damage that comes from this situation is profound. It is a scar across our collective soul.
2014-11-21: When the revolution comes, this will be very useful to smoke out people.
2018-06-07: A New Yorker take
Bullshit, like paper waste, accumulates in offices with the inevitability of February snow. Justification reports: What are these? Nobody knows. And yet they pile up around you, Xerox-warmed, to be not-read. Best-practices documents? Anybody’s guess, really, including their authors’. Some people thought that digitization would banish this nonsense. Those people were wrong. Now, all day, you get e-mails about “consumer intimacy” (oh, boy); “all hands” (whose hands?); and the new expense-reporting software, which requires that all receipts be mounted on paper, scanned, and uploaded to a server that rejects them, since you failed to pre-file the crucial post-travel form. If you’re lucky, bullshit of this genre consumes only a few hours of your normal workweek. If you’re among the millions of less fortunate Americans, it is the basis of your entire career.
2018-09-20: Scott Alexander looks into the topic
I write a note saying: To whom it may concern: I am a psychiatrist treating Mr. Smith. He tells me that he has chronic back pain (“lumbago”), and asks to be allowed to bring in his own chair to work. Yours, Dr. Alexander It’s too soon to have a good sample size. But it seems to usually work. I think it works because there is nobody at Mr. Smith’s workplace – maybe nobody in the entire world – who’s really invested in preventing Mr. Smith from bringing a chair into work. Someone wrote up a procedure for employees using special chairs, so that they’re not the sort of cowboys who make decisions without procedures. Someone else feels like they have to enforce it, so that they’re not the sort of rebel who flouts procedures. But nobody cares.
2021-03-06: It’s surprising that it took so long to automate extremely low value work like copying & pasting, and somehow requiring a college degree:
they do the dull-but-critical tasks that someone named Phil in Accounting used to do: reconciling bank statements, approving expense reports, reviewing tax forms. […] She’d known that her job was straightforward and repetitive, making it low-hanging fruit for automation. The experience was a wake-up call. She had a college degree and was early in her career.
2023-01-17: The race between bullshit jobs and automation
We escaped the original Malthusian trap with the Industrial Revolution, expanding capacity faster than the population could grow. A sufficient lead altered underlying conditions to the point where we should worry more about declining population than rising population in most places.
Consider the same scenario for a potential AI Revolution via GPT-4.
Presume GPT-4 allows partial or complete automation of a large percentage of existing bullshit jobs. What happens?
My model says this depends on the speed of adaptation.
Reverse Engineering Superbugs
You can see that the gene for PBP2_ECOLI has a 100% match inside the genome of O104:H4. Now that we have this list, we can answer some interesting questions, such as “How many of the known drug resistance genes are inside O104:H4?” I find it fascinating that this question is answered with a shell script:
cat uniprot_search_m9 | awk '{if ($3 > 99) { print;}}' | cut -f2 |grep -v ^# | cut -f1 -d"_" | cut -f3 -d"|" | sort | uniq | wc -lThe above script tells us that 1138 genes are a 100% match against the database of 1378 genes. If you loosen the criteria up to a 99% match, allowing for 1 or 2 mutations per gene — possibly a result of sequencing errors or just evolution — the list expands to 1224 out of 1378.
grepping the genome of e.coli for cure clues
The Information
The Information isn’t just a natural history of a powerful idea; it embodies and transmits that idea, it is a vector for its memes (as Dawkins has it), and it is a toolkit for disassembling the world. It is a book that vibrates with excitement, and it transmits that excited vibration with very little signal loss. It is a wonder.
A history of information theory. How civilization acquired software. Very interesting read.
Acclaimed science writer James Gleick presents an eye-opening vision of how our relationship to information has transformed the very nature of human consciousness. A fascinating intellectual journey through the history of communication and information, from the language of Africa’s talking drums to the invention of written alphabets; from the electronic transmission of code to the origins of information theory, into the new information age and the current deluge of news, tweets, images, and blogs. Along the way, Gleick profiles key innovators, including Charles Babbage, Ada Lovelace, Samuel Morse, and Claude Shannon, and reveals how our understanding of information is transforming not only how we look at the world, but how we live.
Prezi
The Zooming Presentation Editor sure this will get old but for now, wow.
Great Stagnation
As Tyler tells the story, there is a progressive expansionary impulse to government, for which technological change creates opportunities, so government expands until those opportunities are fully exploited. Tyrone says his brother has the story backwards. Why, asks Tyrone, does government not only expand in absolute terms as a response to technological change, but also in relative terms? After all, as Tyler points out, private enterprise also has a natural expansionary impulse. With technological change, Tyler writes, “Everything was growing larger.” Yet, to the degree that we can measure it, government has grown dramatically in its share of the overall economy. Why does government win? Tyrone says government is a reluctant adopter of new technology (“Have you been to a government office?”), but that government outgrows the private sector despite this, because the concentration of economic power that attends technological changes demands countervailing state action if any semblance of broad-based affluence and democratic government is to be sustained. Tyrone (who is much more arrogant and less pleasant than his brother) proclaims this to be his “iron silicon law”: In (non-terminal) democratic societies, technological change must always and everywhere be accompanied by the growth of institutions that engender economic transfers from the relatively few who remain attached to older productive enterprises to the many who require purchasing power not only to live as they did before, but also to employ one another in novel or more marginal activities that were not pursued before. Inevitably those institutions develop in state or quasi-state sectors (which include the state-guaranteed financial sector and labor unions whose “collective bargaining” rights are enforced by the power of the state). Tyrone tells me that the only thing the post-Reagan “small government” schtick has accomplished is to push this process underground, so that covert transfers have been engineered by a “private” financial sector in ways that are inefficient, nontransparent, and often fraudulent according to traditional laws and norms. Some of these weak institutions upon which we relied to conduct transfers broke in 2008, so now we’re really feeling the pain. We’ll continue to feel the pain until we restore the ability of the financial system to hide widespread transfers, or until we employ some other sort of institution to provide a sustainable dispersion of purchasing power.
Argues that the state grows because technology disrupts widespread affluence and the state is stepping in to “preserve democracy”.
2012-12-10: See also: cookie cutter “innovators”.

2013-03-03: I like this hypothesis. I have yet to meet a MBA where that fly-by-night degree didn’t count against the person. MBAs are often seen as a miracle cure for ailing careers, or picking the wrong major in college, but really all they signal is that you don’t understand opportunity cost.
The business mentality that focuses on short terms profits is what is preventing the rollout of radical technology. The fault is regulation and regulators. If a company was truly innovating, then that company would outpace the regulators. If a company is moving so slow that they have not escaped the regulatory paradigm then they have not achieved a true moonshot technology. Masses of people with MBAs are managing companies for the last several decades. They focus on milking the profit of existing technology. They can milk a cow but they cannot generate a truly new cow.
2017-03-02: Americans are risk-averse
Americans have stopped taking risks, are too comfortable, and rely too heavily on incremental improvements of existing goods & ideas, which has resulted in a stagnation of our culture and economy.
2019-04-12: We might start to emerge from stagnation
We are now starting to get a hint of the future transformative technologies that you guessed were on their way in “The Great Stagnation”. You had not speculated on what they might be, but there are faint hints on what is likely to happen. I believe this article is one leg: extremely fast air travel. The second leg is the Hyperloop and similar: extremely fast ground travel. The third leg is synthetic biology. The fourth leg is quantum computing, which is finally starting to show that it might work. And the fifth, and final leg, is fusion energy, which looks eerily like it will actually come to fruition this time. Put those 5 together and you have the makings of a new economy, with a huge burst of growth to come for many decades. These are just faint hints, of course, but they’re starting to get increasingly clear.
2020-12-14: Perhaps Covid helps with ending stagnation
If the Great Stagnation is ending (we will see), it seems as if the COVID-forced remote work revolution has to have played some sort of role.
Speaking from personal experience as a white collar Exec, the productivity gains for our highest value workers has been immense. The typical time-sucks and distractions of in-office work have been eliminated, as have their personal time investments like physically visiting the grocery store or running errands. Mental focus on productive efforts is near constant.
Perhaps most importantly, work travel is not happening. Valuable collaborations with colleagues, customers, regulators or other partner companies aren’t delayed by the vagaries of the various groups’ availability to meet in person, navigating being in different cities, flights, hotels, etc. Collaboration happens as soon as you have the idea to meet via Zoom. And a lot more collaboration happens as a result. It may be lower productivity collaboration than meeting in person around a whiteboard (maybe), but the sheer quantity of it means on net there’s perhaps been a boom in cross-pollination of ideas.
2021-01-06: Software stagnation
Software is eating the world. But progress in software technology itself largely stalled around 1996.
2 possible causes come to mind: we’re in exploit, rather than explore mode (there’s an overhang of areas where we haven’t even applied our current software technologies, as the pandemic has demonstrated: Japanese companies still fax, checks are being mailed out, etc). And we have too much software, and don’t know how to replace / refactor it (think of all the systems still running on COBOL / Excel)
2021-11-05: Here’s a longer piece on COBOL:
The problem for banks, though, is that while their COBOL may be stable, their customers’ expectations aren’t. As you probably realize, the landscape of the financial industry is shifting quickly. Transactions are increasingly happening on Venmo-style apps that let people ping money to friends; services like Coinbase let people buy cryptocurrency; there are new lending apps like Tala and Upstart. People now expect ever-easier ways to manage their money via software. This is where banks, which should have inherited advantage in moving money around, have it harder. It’s difficult for them to roll out buzzy new features quickly, because they have to deal with their Jurassic “technology stacks”. Those old COBOL-fueled backends store data in disparate chunks — “they have a lot of silos”. And it’s dangerous, of course, to tinker much with the old code: “You’ve got resource pain, technical pain, operational pain, risk pain.” But a startup can do whatever it wants. There are no old systems. They’re in what programmers lovingly call a “green field” situation. Instead of buying hundreds of thousands of dollars worth of mainframe computers to store and process their data, they just rent space on a “cloud” system, like Amazon’s. They can write code in new languages, so they can hire nearly any eager young computer science student. And they don’t even need to build everything themselves: When Showoff is crafting a new fintech app, it might use an existing service to handle a tricky task — like using Stripe to process payments — rather than trying to create that software themselves.
ffmpegX
if they weren’t so incompetent with library locations, would be a nice ffmpeg frontend for mac.
Cyberwar
Cyber weapons beg to be used, so limits on stockpiles, and restrictions on tactics, are a logical endpoint. International banking, for instance, could be declared off-limits.
You’re Stealing it Wrong
amusing talk on the genesis of the cracker scene. as a byproduct, also the demo scene.