On slop, first thoughts

I saw a manifesto on twitter the other day.

Nothing any of us hasn't seen a thousand times before. The same confident declarations, the same half-understood references name-dropped into a rambling mess, quotes pasted in without the least understanding of historical context. The kind of dreck that's been with us at least since the dawn of the printed page. That's not new.

The AI capstone image summation though - that was new. DALL-E couldn't make the manifesto coherent, but it could make it pretty.

And still my finger clicked "mute" even before my mind registered the fourth sentence.

Two days later, I'm still thinking about the experience. Partly the usual - "how long will we be able to tell?" I'm somewhat less bothered on that front, to be honest. Leaving aside the biases of the models themselves, a low-capacity ideologue trying to talk a model into affirming his or her preconceptions is swimming against the tide. By the time the derp has been massaged by a thinking machine into something halfway rational - is it still derp?

More personally though... I'm so wary of making the same blind mistakes myself. AI can deliver such a convincing illusion of self-competence, and I'm not just talking about the sycophancy. Having a work product appear in front of you comes with the mental experience of having caused it to be made. But designing is not building, and describing is not designing. I don't think our brains can really accommodate that distinction yet, not at the base experiential level. At least mine doesn't.

In our manifesto writer it was easy to spot - the writer him or herself clearly had no firm foundation. The gulf between what the writer knew enough to ask for and the resulting work product was so substantial that the effect was... unconvincing, to be kind.

... but I also know I could well find myself in the same position, or near enough.

I've got the foundations of bioinformatics down, but not the graduate level expertise of molecular bio, cellular chemistry, physics down at protein scale... I've got decades of in-the-trenches enterprise software building, but I'm not a comp-sci professor and now I'm swimming in novel architectures and reading up on cutting edge hardware.

I am constantly working well out over my skis, and I know it.

Some of that means human consultation, and some... well, that's just part of working at or near a frontier. You can't know what works until you try it.

So back to work I guess. And more time in the books.

Authenticity is hard.