My Boy Builds Coffins

My boy builds coffins with hammers and nails
He doesn't build ships, he has no use for sails
He doesn't make tables, dressers or chairs
He can't carve a whistle because he just doesn't care
    -- Florence and the Machine, "My boy builds coffins"

Carpentry is a pretty versatile skill; we haven’t yet stopped having use for wood. Not all carpenters are equally versatile. Some will turn their hands to anything. Others are like the coffin-builder, masters in one domain with no interest beyond it.

It isn’t just a matter of skill. Sure, a cabinet-maker has skills beyond those of a joiner. But if Florence’s boy went to the unemployment office, they’d push him towards, say, a job building tables. On a purely technical level, they might well be right.

Every domain has not just its own set of skills, but its own aesthetic, its own community of practice, its own motivation. If you get your joy building coffins, what’s the appeal of a chair?

The last decade of AI progress has wiped out any number of academic and creative niches. Decades of work in Natural Language Processing, for instance, is obsolete because GPT can do it better.

Progress tends to merge existing disciplines into a blob, before eventually spawning new offshoots. In the aftermath of a big paradigm shift like this, a few naive techniques beat the state of the art across many disciplines. Ten years from now, various areas will have learned how to layer their own skills on top of the basics. And with that, specialization and communities of practice will re-emerge, albeit not in quite the same constellations as before.

I’m writing this as somebody whose expertise has not been made obsolete by AI – I avoid that simply by not having such deep expertise in the first place! Yet even vicariously I feel a certain amount of grief for the work and skill confined to history. And even when that is a shared experience – I’m not aware of any cultural structure to recognize and channel that grief.

Quickly choosing software libraries

One of the rarely-made-explicit skills of a developer is choosing a software library for some task. Just now, I realised I’ve built up some habits of how I do this, without ever making it explicit. Here’s what I’m generally doing:
– find one package that does the job. perhaps by straight-up googling for it, perhaps by checking stackoverflow
– search Hacker News discussions about it, using duckduckgo’s ‘!hn’ bang command. Here I’m trying to get a general idea of how people see this library, how alive it is, and especially what people are using instead. I’m looking through
– top-ranked posts: generally the launch and major releases, sometimes somebody complaining about it
– top-ranked posts from the last year: gives me a sense of the direction. Quite often you’ll find a low-comment post asking ‘what’s up with X?’, and somebody in the comments saying ‘we’re all using Y instead’
– comments: will generally turn up some where the library is one of a list of options (try X or Y or Z), or where somebody is arguing when/when not to use it
– Through this I build up a list of maybe 2-4 libraries in the same area, and an idea of how they are seen. Then for each of them, check:
– the homepage, especially the quickstart tutorial
– github, or wherever the code is. number of stars/forks, maybe glance at the open issues
– Try the most-promising looking one. If I hit a roadblock, go back and look at number 2

In most areas, this is sufficient to quickly find a ‘good enough’ solution. Then I can forget about it, and focus on the 10% of cases where I need to have higher standards

Command Palette Shortcuts

Quick reference for myself: the shortcut to open the command palette (aka heads-up display) in various programs:
– intellij idea: C-S-a
– VS Code: C-S-p
– obsidian: C-p
– KDE: M-
– Github: C-k
– Libreoffice: S-

Not yet implemented (but coming): kate, krita

Background: The command palette is my favourite UI innovation of recent years, one I already find it hard to live without. One shortcut brings up a searchable list of commands, which permanently removes the hassle of browsing through menus. But every program has chosen its own shortcut to bring up the palette, and I find them hard to remember. Hence this cheat-sheet

[shortcuts are listed emacs-style: M is alt, C is ctrl, S is shift]

Sentences I have enjoyed

Sentences I have enjoyed:

Mauritius, arguably Africa’s most successful economy, has outsourced its final court of appeal to Britain’s privy council, improving its reputation for rule of law and becoming a successful offshore financial centre in the process
–[How to create a city | Financial Times ]

We begin with a striking fact: The defendant’s face alone matters greatly for the judge’s jailing decision. In fact, an algorithm given only the pixels in the defendant’s mugshot accounts for up to half of the predictable variation.
–[Machine Learning as a Tool for Hypothesis Generation – Marginal REVOLUTION ]

apparently the salinity of our bodies matches that of primordial seas, so in a sense we never really left the oceans. Our micro-organic aquatic ancestors simply constructed meatbag spaceships with artificial life-support aquatic environments inside to explore beyond their oceanic home world.
–[Salt-Seeking ]

A banking system is a superposition of fraud and genius that interposes itself between investors and entrepreneurs
–[interfluidity » Why is finance so complex? ]

Where I’m following AI news

Bracing against this year’s endless future-shock, I’ve still been maintaining my “Awesome Prompt Design” collection of links.

But when so much is happening day by day, it’s a struggle to keep any kind of perspective. So here are the sources I’ve been finding most useful for filtering the firehose:

  • Simon Willison has the perspective of a skilled programmer without an AI background. So his attempts to understand and explain things fit appropriately into my level of knowledge
  • Nick St. Pierre is pushing the boundaries of image- and now video- generation, especially good on AI. Best of all, he gives crystal-clear twitter-thread explanations of what he’s doing, in enough detail to follow along at home
  • Zvi Mowshowitz (thezvi) is smart and reads everything. Unfortunately he tends to also write everything, making his AI Roundups a challenge to get through
  • The Neuron is a bare-bones daily email covering the big releases

Learning Languages with GPT

I’ve been mentally replaying how different my language-obsessed childhood would have been if GPT had been around then.

Even now, while my head says “you don’t need to learn more languages”, it’s hard to stay away from such a shiny new thing.

So here are some of the ways I’ve been using GPT for langauge study

Conversation

Moving general-purpose interactions with GPT into the language I’m learning is a good step towards immersion. It’s a big step up from just reading foreign-language materials online – this way I’m forced to compose full-sentence queries in my target languge.

Ideally, GPT would correct my text while I’m doing this. Alas, it tends to abandon the language-tuition side-quest and focus in on whatever else we are talking about. This fits a general pattern where GPT is pretty bad at balancing multiple tasks within a single conversation

Where I don’t know a language well enough to talk to GPT in it, I can ask it to fill in the blanks. I start off with a prompt like this:

If I write in English in quotation marks (“like this”), you should translate what I write into simple Turkish. If I write in Turkish, you should correct any mistakes and reply in simple Turkish. If I write in English without quotation marks, you should reply in English

Here is an alternative prompt in a similar style.

Explanation

GPT does a reasonable job of explaining grammar:

Please explain the form and meaning of the past definite verb in Turkish. Give examples

Of course, there’s not much advantage of doing this over looking in a textbook. Where GPT shines is in explaining words in context, such as explaining a line from a Turkish pop song:

gloss the meaning and grammatical form of each word in this line: Dedim ama kalbim sanki karakolda

  • Dedim: “I said” – past tense of the verb “demek” (to say)
  • ama: “but” – conjunction used to indicate a contrast or contradiction
  • kalbim: “my heart” – noun (heart)
  • sanki: “as if” – conjunction used to indicate a comparison or resemblance
  • karakolda: “in a police station” – noun phrase (police station)
    Overall meaning: “I said it, but my heart feels as if it’s in a police station.”

Flashcards

I use Anki to memorize vocabulary. It’s a flashcard program with spaced repetition. That means it tries to quiz me on each word while I can just remember it, which helps the knowledge to burrow ever-deeper into my mind.

I can use ChatGPT to generate content for flashcards. Anki accetps

I will paste text in Turkish. You should create a glossary of all words used in the text, with English translation. The glossary should be in csv format: each line should be formatted as the Turkish word, then a comma, then the English translation. The lines should be arranged in order of the first appearance of the Turkish word in the text.

I’ve found it hard to prevent GPT inserting a space after the comma in this output format, and decided to just live with that.

I’d only recommend GPT’ing flashcards where you have some special need, such as vocabulary from a particular text or on a particular topic. If you just want a general word-list for a language, the hand-crafted shared decks available on the Anki website are much more reliable.

Language games

Squidgies is an app for GPT-enhanced language learning. They’ve done us a favour by releasing the prompts they use to generate language games and class activities. Things like:

Generate a list of questions for someone learning English to practice speaking. Topic: eating in a restaurant

or:

The objective of this game is for the player to have their partner guess the word they’re thinking of.
Correct grammatical errors and non-English text with CORRECTION on a new line. Accept partial sentences.
If the student’s response does not make sense, ask him/her to clarify what he means.

Scenius

An article on why so much music comes form Sweden:

Sweden by accident created unusually good conditions for musical scene creation.
What is a scene? It is a group of people who are producing work in public but aimed at each other. [Swedish bands] performed on stage — but the audience was mainly their friends who played in other bands. If they were anything like the other local scenes I’ve seen, they challenged and supported each other to be bolder, more ambitious, better.

These scenes in turn were enabled by policies that gave Swedish kids free music lessons, with the side-effect of providing young musicians with plenty of jobs teaching music.

Something similar happened in the UK in the 80s and 90s, now imagined as a golden age of making music while claiming the dole. Thatcher inadvertently helped this along. Her Enterprise Allowance Scheme, intended to help the unemployed start new businesses, became a standard route for anybody trying to make a musical career.
So just about any band which got started in the 80s or 90s – from Oasis and Pulp all the way down – spent some time on benefits at the start.

Links and Snippets, AI Edition

How to get GPT to take your existing documents and code into account, according to Simon Willison. You use something else as a first-pass search. That turns up a few relevant documents, which you feed into a GPT prompt and let it work its magic. GPT Index is a more systematic implementation of the same idea.

For a flickering moment, Deepmind released dramatron, a tool for collaborative AI/human scriptwriting. Then they almost immediately shuttered it, leaving nothing but vague PR.

FactGPT claims to be a GPT that’s aware of current events

Style2paints is a tool for colorizing line art. The new (not-yet-released) version has 2 gremlins inside. Dorothy is a good girl who will precisely follow your input sketch. Alice will try to make something good, regardless of how crappy your input sketch might be.

Links and Snippets

Geschwind Syndrome is a form of religion where you turn religious and can’t stop writing.

“We estimate that on average 10% of large publicly traded firms are committing securities fraud every year”. The approach is to look at former clients of Arthur Andersen, a big accounting firm which collapsed in scandal in 2022. The idea is that their former clients were more thoroughly examined by the new auditors, dredging up fraud which might otherwise have remained hidden. They find that two thirds of fraud is not detected.

A list of tech coops

Argument that if AI is going to conquer the world, interest rates will rise first

ChatGPT – scary but exciting

ChatGPT leaves me slightly more excited than terrified, but it’s a close-run thing. Although OpenAI so far haven’t released much detail on how ChatGPT, I assume it’s brodly similar to earlier GPT projects. That is, it’s one model that has learned to predict text by reading the internet, then been fine-tuned to function in Q and A. ChatGPT would seem much less scary/exciting if it turned out to be a stitching together of modules trained specifically to compose poetry, teach programming, and so on.

Until now, I’d been relatively pessimistic about Artificial General Intelligence, at least compared to the people most actively talking about it. I’d figured it would eventually happen if we managed not to destroy the planet, but not in my lifetime. I’d expected that we’d hit a ceiling with the current generation of pretty simple architectures, then have a period of increasing specialisation within domains before the next quantum leap.

Between ChatGPT and Stable Diffusion, though, it seems that AI is advancing far more quickly than society has any hope of adapting to deal with it. It’s not quite the singularity, but the next 20 years seem set to be Interesting Times.

A few of the snippets I’ve pulled from the torrent of people doing amazing things with ChatGPT: