Is SOLID a data format? Or just a server that serves different formats of data through a consistent API? I've glanced through the documentation and I'm not clear on exactly what it's doing that makes it different.
Solid is a sort of poor way of doing what it wants to do. The idea is to cleave the data from the application so that the user keeps their data and offers applications access to it. But this is a naive choice, in the same way the Internet was, perhaps!
First, most data is not useful without the application that created it. One of the most ironic issues with Google Takeout is you can download data for all of your Google services but import it into almost nothing, there's a handful of nerd projects that can import some of it but most of it requires you be a programmer yourself to use. So the user really should own their data and the app needed to use it.
Second, as soon as a company-hosted app can access your data it can copy it. So keeping your data separate in a pod isn't meaningfully helpful for privacy either. If you are connecting to their servers to use an app, it can be proprietary and you can't see what it does with your data, so you can assume it isn't private anymore.
There are a lot of better implementations than Solid. But since Tim invented the Web, his implementation continues to get press on major news outlets even though it's a bad idea that's gotten no traction in over a decade.
I contribute to a project called Sandstorm where the app data is directly associated with apps you can install on the server. I think it has the best approach for a lot of reasons, both in terms of security and making it easy to use if you are not a developer.
But almost any self-hosted app platform suitably exceeds the assurances Solid can meaningfully provide.
We're living in a reality where nuclear waste was not always properly contained and the repercussions of it are long-lasting. While this particular case is related to the Manhattan Project, it's causing all types of issues to this day:
I visited Auschwitz-Birkenau in February of 1995. It was well below freezing and there was some type of ice ball precipitation, perhaps because it was too cold to snow. I was the only person there.
I walked all the way back from the famous entrance gate, along the train tracks, to the monument at the back. The place was huge and imagining people suffering there during that type of weather was especially heartbreaking. I was luckily able to convince the taxi driver to wait for me. I have some black and white photos I took of it somewhere on my shelves. That visit sticks with me more powerfully than almost anywhere else I've been.
I've tried using this before and pretty quickly migrated away. While I like the idea of it, the default styling is not great and I felt frustrated with the amount of adjustments it needed to support a data-dense site.
I loaded this same IMDB dataset into a Kuzu DB the other day, just for fun. It was an interesting project but I would agree that this dataset in particular lends itself to a graph database. Kuzu started to choke with > 10M nodes & 10M edges but I was doing a naive insert & probably could fit the entire dataset in without trouble using their direct CSV import.
Interesting points here. I’ve found the “graph DB vs. relational DB” discussion usually gets framed as an either/or, but there’s a middle ground.
A lot of teams already have their data sitting in Postgres, Mongo, or a lakehouse. Spinning up a separate graph database just for traversals often means duplicating data, building pipelines, and keeping two systems in sync. That’s fine if you need deep graph algorithms, but for many workloads it’s overkill.
What some folks are exploring now is running graph queries directly on top of their existing data, without having to ETL into a dedicated graph DB. You still get multi-hop traversal and knowledge graph use cases, but avoid the “yet another database” tax.
So yeah...graph databases are great, but they’re not the only way to model or query graphs anymore.
I went down this road with Clickhouse and spent 18 months setting it up for realtime analytics. I'm not sure it's going to stick, primarily because our Data Transformation & Aggregation steps require some gnarly joins across large tables and Clickhouse does not handle large joins well. The consequence is that the aggregation has to happen in a separate system (currently using Snowflake) and when there were changes to what we were processing, it sometimes requires gymnastics both in the aggregation layer and inside of Clickhouse to accommodate the change. Denormalizing was rife with tradeoffs, mostly just to make Clickhouse happy. On top of that, we leaned heavily on projections for performance, which is wonderfully automated, but also meant waiting for unpredictable background processing during backloads, etc.
We might stick with Clickhouse, but after working with it for a year and a half, I'm curious to see whether a system that handles joins more effectively would be a better fit. To that end, my next R&D project is to set up a vertical slice of our analytics on Apache Doris to see how well it handles a similar workload.
Disclaimer: I'm the product manager for ClickHouse core database.
Which version are you working with? I recommend trying the latest version. For the last eight months, we have spent considerable time improving our JOINs.
Sai from ClickHouse here. Adding to above, we just released a blog that presents JOIN benchmarks of ClickHouse against Snowflake and Databricks. This is after the recent enhancements made to the ClickHouse core. https://clickhouse.com/blog/join-me-if-you-can-clickhouse-vs.... The benchmarks is around 2 dimensions of both speed and cost.
This is really encouraging! Commented elsewhere in the thread but this was one of the main odd points I ran into when experimenting with ClickHouse, and the changes in the PR and mentioned in the recent video about join improvements (https://www.youtube.com/watch?v=gd3OyQzB_Fc&t=137s) seem to hit some of the problems. I'm curious whether "condition pushdown" mentioned in the video will make it so "a.foo_id=3 and b.foo_id=a.foo_id" doesn't need "b.foo_id=3" added for optimal speed.
I also share nrjames's curiosity about whether the spill-to-disk situation has improved. Not having to even think about whether a join fits in memory would be a game changer.
[Edited to add: looking at comments from
saisrirampur and SnooBananas6657, ClickHouse has worked some on joins recently, and there's a very recent a PR around optimizing join order. Cool!]
In playing a little with ClickHouse, issues with joins were one of the biggest things that jumped out at me.
It wouldn't change the join order to try to ensure the smaller side was in RAM. It wouldn't propagate WHERE conditions to the other side of a join, so "WHERE a.customer_id=3 AND b.customer_id=a.customer_id" wouldn't automatically filter the scan of b to blocks where b.customer_id=3. Projections could theoretically be used to speed joins on the projection's ORDER BY columns, but it didn't appear to happen. You'd sometimes need to manually set the join algo for best performance, when I'd rather the optimizer choose it, maybe adaptively based on whether it seemed to make sense to do a join in-RAM.
ClickHouse was also great at certain things, like scanning a ton of rows very efficiently and interfacing with lots of systems (its table functions could pull from lots of sources). It was easy to pick up and start playing with, and seems to have found use in a lot of organizations and at substantial scale. And there are a lot of features that look potentially quite powerful (incremental materialized views, aggregating/deduping merge tree variants, etc.). The best way I could process what I saw experimenting with it was that they'd focused on a providing really efficient low-level execution engine, but sometimes you needed to act as the query optimizer.
I got a recent ClickHouse and played with it, and looked around GitHub some; replying to this dead thread to update outdated bits of the above.
In general: given a huge hash join, you can shrink it (reorder, etc. so fewer rows would need hashed), sort (take advantage of sort order for an efficient join), and/or spill it to disk. ClickHouse's improvements from 2024.12 forward have made it much better at the first of the three; it could get much further with the other two.
1) Shrinking -- "Can we make it a small join?" They've done a lot to solve this one! They'll swap table order, propagate WHERE conditions, and end up using projections, so quite often if the join can be small, it will be, without special effort on the query writer's part. To be clear, this is a huge win for many use cases, and I'm not writing more because all I can say is "great."
2) Sorting -- "Can we take advantage of an existing sort order?" Huge joins can be efficient and not run out of RAM if the data is already ordered by the join key and all you have to do is walk through part of it in order. ClickHouse doesn't automatically do this; it defaults to hashing even when the tables' natural row ordering would make sort-based joins fast. Docs suggest users consider full_sorting_merge when tables are already ordered by the join key. Perhaps a future setting could make the optimizer consider it.
Joins with coinciding ORDER BYs seem like they could occur naturally in "might-have" situations, e.g. joining pageviews to conversions on a shared key.
2a) Sorting with projections -- "Can we use sort orders if we include projections?" If rows aren't already on disk in the correct order to make a sort join fast, the ClickHouse administrator could add a projection with a different ORDER BY to one or both tables to fix that! Right now ClickHouse will pick projections to aid in filtering, but won't look for projections that allow a fast sort join.
Combining 2 and 2a, ClickHouse admins would gain a pretty big hammer to use on problem joins: create a projection or two allowing a fast sort-join, and now the former problem query is efficient with no effort on query authors' part. This is valuable when the admins are willing to understand query and execution patterns precisely, but not everyone (or everything, when some queries are generated) can write queries with optimization in mind.
Query authors can force use of a projection with the mergeTreeProjection function, and force join_algorithm = 'full_sorting_merge', but that's the user acting as query optimizer, not the desired end state.
3) Spilling -- "Will we automatically spill to disk instead of having an out-of-memory error?" Looks like this is still a "no". Docs show options to stop joins at a certain size, but I don't see a way to e.g. trigger a retry with a more spill-friendly join_algorithm when a join gets too big (or if the planner expects it will be big). There is a very interesting patch in https://github.com/ClickHouse/ClickHouse/pull/72728 which introduces the idea of query-wide RAM accounting that can trigger spill to disk, but it's only used in grace_hash as of that PR.
--
I haven't really looked at DISTINCT or aggregation's RAM use or whether they take advantage of sorts. And maybe the better path is to make transparent spill-to-disk something you can rely on then reduce how often you need to spill as an optimization. Either way, ClickHouse could get from its current state--where joins that can fit in RAM are efficient but larger ones need some hand-holding--to a state where you really can throw all sorts of queries at it and reliably get results without really having to guide it.
+1 to StarRocks. You can cut out a lot of the weight associated with denormalization, which ClickHouse almost forces you to do. Crazy big cluster sizes as well
In our experience, the primary driver of Snowflake costs is not the compute for aggregation, but the compute required for lots of reads/scans.
We recently built a Snowflake-to-ClickHouse pipeline for a customer where aggregates are built hourly in Snowflake, then pushed into a ClickHouse table to power their user-facing dashboards.
By offloading dashboard queries to ClickHouse, they slashed their Snowflake bill by ~25%, which was worth millions to them.
(Admittedly, running aggregations elsewhere—for example, in Spark—could further reduce costs, but you would then need Iceberg to make the tables queryable in Snowflake.)
I'm in an enterprise environment where a central IT platform team controls what size warehouses we can have in Snowflake. They are not receptive to arguments for larger warehouses, unfortunately. Our issue becomes long-running queries b/c Snowflake spills the data to disk during the joins. TBH, I could join the data more quickly on my laptop than in the warehouse I'm allowed to use. Anyhow, I have then an old build server that is beefy & has 512 GB of RAM, so I can set up my aggregation and/or OLAP services there, since it's an unencumbered playground.
I was playing with Aseprite (pixel editor) the other day. You can script it with Lua, so I asked Claude to help me write scripts that would create different procedurally-generated characters each time they were run. They were reproducible with seeds and kind of resembled people, but very far from what I would consider to be high quality. It was a fun little project and easily accessible, though.
I've been looking for a good pixel-art AI for a while. Most things I tried look okay, but not stunning. If anyone has had good experience with an AI tool for that, I'd be grateful for a link.
Palantir was 90% about ontology creation back when I first used it in 2009. They knew at that point that without some level of structure that mapped to the specific business problem that the data input into the graph was difficult to contextualize.
That's because India has to. Domestic demands are huge and India's coal isn't very high in quality. Not to mention coal power is largely state-controlled and doesn't allow for much private frolicking.
It's quite the opposite situation than the US, where coal is extremely high-quality and private player participation is unrestricted.
Almost all new energy construction is non-coal. Coal has collapsed even here in the US, and the current administration is unlikely to seriously change the trajectory. Gas is increasing, but mostly here in the US, but production is dropping again.
This is technically impressive and I commend the team that brought it to life.
It makes me sad, though. I wish we were pushing AI more to automate non-creative work and not burying the creatives among us in a pile of AI generated content.
I think the non-creative work is coming... but it's harder, needs more accuracy, and just generally takes more effort. But it's 100% coming. AI today can one shot with about 80% perfection. But for use cases that need to be higher than that, that last 20% is grueling to gain. It's like taking a jet across the country, and then getting jammed in traffic while you're taking a taxi to your hotel.
The amount of gatekeeping I see when this topic is brought is outstanding! Why can't people be happy that more individuals would be soon able to create freely in a more accessible way?
Personally I can't wait to see the new creative doors ai will open for us!
How is the requirement to use a computer and maybe pay a cloud subscription in the long term more accessible than other kinds of art? Which individuals are gatekept exactly? Before you bring up disabled people (as often happens when the term accessibility is used), know that many of them are not happy to be used as a shield for this without ever being asked and would rather speak for themselves.
I've tried AI image generation myself and was not impressed. It doesn't let me create freely, it limits me and constantly gravitates towards typical patterns seen in training data. As it completely takes over the actual creation process there is no direct control over the small decisions, which wastes time.
Edit: another comment about a different meaning of accessibility: the flood of AI content makes real content less accessible.
>How is the requirement to use a computer and maybe pay a cloud subscription in the long term more accessible than other kinds of art?
Because many other kinds of art require thousands of hours to learn before getting to the level of current AI
The real gate keeper to art isn't the cost of a pencil, it's the opportunity cost of learning how to use it
Some people have creative ideas they cannot realise and tools like AI help them do it. The more people that can realise their creative ideas the better it is for everyone.
so its for lazy people? who don't want to learn a skill? there are many ways to realize your creativity, and now you have to write your "prompt" for your creative result? then why not just write a story? like authors have been doing for ever ?
>The more people that can realise their creative ideas the better it is for everyone
There is a fundamental misunderstanding of creating going on here. There's a reason why people always talk about the work/journey/process rather than the end goal. That's what makes someone an artist--not the end result.
do you think taking a photo of something is skipping to the end goal of painting? no. it will just be used as a tool for more creativity for the people who will do the work.
>do you think taking a photo of something is skipping to the end goal of painting
Going outside and taking a photo requires you to be engaging with the scene around you. You are the actor doing the thing. When you prompt an AI, the AI is the actor doing the thing.
Yes! Directors orchestrate hundreds of people and other creatives, have a say in how scenes are shot, how actors perform their lines, which music gets used, which lines need to be changed or added, etc.
Not to mention that many directors also write the script for the film.
Wouldn't all this hold true for AI videos as well, it's just that you'll be going through hundreds of prompts and bad generations before you get to something approaching your vision?
N.B. If directing people is the distinguishing factor, then animation directors are robbed of their claim to artistry as well, as is basically any solo artist in history
I never said directing people is the distinguishing factor. I only said a director is an artist as they are deeply involved in the creation of the art in question. There is no hidden layer of linear algebra that obfuscates the creative process and that does the majority of the work.
this view is really limited. like I said, people who will do the work... will. You are thinking of the majority of lazy people who aren't producing anything of real value in the design/art world already.
I don't care at all about the philosophy of art. I have creative goals in mind and not enough time to become competent at all the prerequisite skills to master them.
> How is the requirement to use a computer and maybe pay a cloud subscription in the long term more accessible than other kinds of art?
Sometimes I feel like HN comments are working so hard to downplay AI that they’ve lost the plot.
It’s more accessible because you can accomplish amazing storytelling with prompts and a nominal amount of credits instead of spending years learning and honing the art.
Is it at the same level as a professional? No, but it’s more than good enough for someone who wants to tell a story.
Claiming that computer access is too high of a bar is really weird given that computers and phones (which can also operate these sites) are ubiquitous.
> Edit: another comment about a different meaning of accessibility: the flood of AI content makes real content less accessible.
No it does not. Not any more than another person signing up for YouTube makes any one channel less “accessible”. Everyone can still access the content the same.
It would have cost millions. Now one person can do it with a laptop and a few hundred dollars of credits a month.
AI is 100% making filmmaking more accessible to creative people who otherwise would never have access to the kind of funding and networks required to realise their visions.
Since 10+ years ago various YT channels have been creating interesting content, sometimes on less than a normal salary, filmed with a phone and with one person acting for three characters at once. And imho they (fortunately) won't disappear if what you linked is representative for what AI enables.
None of the videos I've clicked on required AI for the content to be good, and some of the randomness has no real reason to be there.
Most of those "skits" can be done on a cheap camera with a single person being the actor. In fact, many such videos existed already in the past.
Also, they're painfully unoriginal. They're just grabbing bits that The Onion or shows like Rick & Morty have been doing and putting a revolting AI twist to it. It screams to me of 0 effort slop made for the sole purpose of generating money from morons with no creativity clicking on it and being bemused for 10 seconds
> How is the requirement to use a computer and maybe pay a cloud subscription in the long term more accessible than other kinds of art?
Accessibility -- and I don't mean this in the sense particular to disability -- is highly personal; its not so much that it is more accessible, as that it is differentlly-accessible.
> I've tried AI image generation myself and was not impressed. It doesn't let me create freely, it limits me and constantly gravitates towards typical patterns seen in training data. As it completely takes over the actual creation process there is no direct control over the small decisions, which wastes time.
No offense, but you've only tried the most basic form of AI image generation -- probably something like pure text-to-image -- if that's what you are finding. Sure, that's definitiely what the median person doing AI image gen does, dumping a prompt in ChatGPT or Midjourney or whatever. But its not all that AI image generation offers. You can have as much or as little control of the small (and large) decisions as you want when using AI image generation.
There's a pretty standard argument that creating artworks should or must require the hardship of developing the skill to bring vision into reality, or paying someone who can. That can be debated, but the position is textbook gatekeeping.
Other disapproval comes from different emotional places: a retreading of ludditism borne out of job insecurity, criticism of a dystopia where we've automated away the creative experience of being human but kept the grim work, or perceptions of theft or plagiarism.
Whether AI has worked well for you isn't just irrelevant, but contrarian in the face of clear and present value to a lot of people. You can be disgusted with it but you can't claim it isn't there.
Gatekeeping means restricting power or opportunities.
Is it gatekeeping when a skill can be learned for free, and on top is also optional?
Is it also gatekeeping to forbid students from using ChatGPT for homework because it would force them to go through the hardship of earning an education?
I have seen what 99% of people are doing with this "clear and present value". Turns out when you give people a button to print dopamine they probably aren't going to create the next Mona Lisa, they're just going to press the button more. Even with AI, creating compelling art is still a skill that needs to be learned, and it's still hard. And why would they learn a skill when they just decided against learning a skill? Incentives matter, and here the incentives massively favor quantity over quality.
> Whether AI has worked well for you isn't just irrelevant
My point was that it's creatively restrictive, and that current models tend to actively steer away from creative outputs. But if you want to limit yourself to what corporations training the models and providing the cloud services deem acceptable, go ahead.
I stated exactly what the enumeration of arguments that I know of are and which one is gatekeeping. (It's still the first one, devaluing or dismissing work out of hand if AI was involved, regardless of other merit.)
Remember how there were all those cake shows all of a sudden, and they were making cakes that looked super pretty, but they were just fondant and sheet cakes? We're not thrilled having to wade through the AI equivalent.
> Why can't people be happy that more individuals would be soon able to create freely in a more accessible way?
The gates are wide open for those that want to put in effort to learn. What AI is doing to creative professionals is putting them out of a job by people who are cheap and lazy.
Art is not inaccessible. It's never been cheaper and easier to make art than today even without AI.
> Personally I can't wait to see the new creative doors ai will open for us!
It's opening zero doors but closing many
---
What really irks me about this is that I have _seen_ AI used to take away work from people. Last weekend I saw a show where the promotional material was AI generated. It's not like tickets were cheaper or the performers were paid more or anything was improved. The producers pocketed a couple hundred bucks by using AI instead of paying a graphic designer. Extrapolate that across the market for arts and wonder what it's going to do to creativity.
It's honestly disgusting to me that engineers who don't understand art are building tools at the whims of the financiers behind art who just want to make a bit more money. This is not a rising tide that lifts all ships.
That's how human brains work. People have an intrinsic need to sort, build hierarchies and prioritize. Effort spent is one of viable heuristics for these processes.
> Why should being an artist be a viable job?
Art itself has great value, if it weren't, museums, theaters and live shows wouldn't exist.
> Would you be against technology that makes medical doctors obsolete?
The analogy doesn't work. The results of a medical process is a [more] healthy person. The result doesn't have any links to the one performing it. Result of an artistic creative process is an art piece, and art is tied to its creator by definition.
I think the result of an art piece is "wow that looks pretty cool / makes me feel this certain way". Human properties aren't required there. Sure, some art rests on there being human properties, but then AI wouldn't be able to replace it by definition. You can argue that AI lowers the discoverability of art that falls in the latter category, but I'd say that it's a solvable problem that can be fixed with better recommendation algorithms.
> I think the result of an art piece is "wow that looks pretty cool / makes me feel this certain way".
This is an expansive definition and thus not useful, because it would include:
1. Natural phenomena (sand on a vibrating plate is pretty cool).
2. Folk crafts (this hand-woven rug sure ties the room together!).
3. Advertisement.
4. Industrial design (this soap dispenser looks like a droid head, awesome!).
5. Drug induced experiences.
6. Art forgery and plagiarism.
Nothing in the list is really art. Rough definition of art is an intentional process (or the results thereof) of self-expression, and/or interpretation/modeling of reality performed with symbolic means. This implies intentionality and a conscience, which current "AI" doesn't have.
> AI lowers the discoverability of art that falls in the latter category, but I'd say that it's a solvable problem that can be fixed with better recommendation algorithms.
Theoretically it is. However, it won't be ever solved and implemented widely due to the lack of incentives and the fact that just replacing it all with "AI" is much more profitable and exploitable.
I would be against technology that freezes medicine at our current understanding and makes it economically unviable to develop new medicine.
I don't think it's worthwhile to explain the inherent value of human created art or that to learn how to do it one must put some effort into it. All I can say is, if you are one of those people who do not understand art, please don't build things that take away someone else's livelihood without very good reason.
I don't think the majority of AI generation for art is useful for anything but killing artists.
On effort being a requirement: part of art is around playing with limits of a medium and finding a way in it, it takes a lot of trials, attempts, and errors for an artist to make their way. It's not a requirement per se, but something needed for someone to intent something different. Not worried about creative ways where artists explore AI, new things will come out of it and it's going to be interesting. Not worried either about post-modernists who already dropped requirements long time ago and tape bananas to walls, they'll find their way. But the category artists who make their way through the effort put in a medium, not only the narrative around the medium will be affected.
On jobs: craftsmanship is slightly different than art: industries are built with people who can craft, there is today an artistic part in it but it's not the essence of the job: the ads industry can work with lower quality ads provided they can spam 10x. There is however an overlap between art/craftmanship: a lot of people working in these industries can today be in a balance where they live with a salary and dedicate time to explore their mediums. We know what will happen when the craftmanship part is replaced by AI, being an artist will require to have the balance in the first place.
It feels like a regression: it leads to a reduction of ideas/explorations, a saturation of the affected mediums, a loss of intent. Eager to see what new things come out of it though.
I think effort is a signal that whatever the thing is, the artist/team -really- wanted to put it out there and have people view it. They had -something- that drove them to take the time and effort to make the thing.
Zero-effort output generators like prompting means people are just generating trash that they themselves don't even care about. So why should I take my time to watch/experience that?
The whole "GenAI is accessible" sentiment is ridiculous in my opinion. Absolutely nothing is stopping people from learning various art mediums, and those are skills they'll always have unlike image generators which can change subscription plans or outright the underlying model.
Absolutely no one should be lauding being chained to a big corp's tool/model to produce output.
---
Why should being an artist be a viable job? Well, people should get paid for their work. That applies to all domains except technical people love to look down on art while still wanting to watch movies, well produced youtube videos, etc. You can see it in action here on HN frequently: someone will link a blog post they took time to write and edit... and then generate an image instead of paying an artist. They want whatever effect a big header image provides, but are not willing to pay a human to do it or do it themselves. Somehow because it's "just art" it's okay to steal.
---
If tech has progressed to the point of true "general artificial intelligence", then likely all jobs will be obsolete and we're going to have to rethink this whole capitalism thing.
I think all industries should be utilizing tech to augment their capabilities, but I don't think actual people should be replaced with our current guesstimator/bullshitter "AI". Especially not critical roles like doctors and nurses.
There's nothing creative in having someone or something else doing the work for you.
"Creating" with an AI is like an executive "inventing" the work actually done by their team of researchers. A team owner "winning" a game played by the their team.
That being said, AI output is very useful for brainstorming and exploring a creative space. The problem is when the brainstorming material is used for production.
The first two paragraphs of your argument could be used to discuss whether Photography (Camera is doing most of the work) or Digital Drawing (Photoshop is doing most of the work) are art.
Both things which were dismissed as not art at first but are widely accepted as an art medium nowadays.
I see this comparison to a camera a lot but I don't think it works (not that you're saying this, I'm just contributing). I'm not an expert but to me the camera is doing very little of the work involved in taking an artistic picture. The photographer chooses which camera to use to get a certain effect, which lenses, the framing, etc. All the camera is doing recording the output of what the person is specifying.
I think there's a sliding scale in both cases. Vanilla prompting something like DALL-E 3 and uncritically accepting what it spits out is the AI equivalent of dime-a-dozen smartphone snapshots of the Eiffel Tower or an ocean sunset. But like your description of professional photography, there are more intricate AI approaches where an expert user can carefully select a model, a fine-tune/LORA, adjust the temperature or seed, inpaint or layer different elements, and of course have the artistic vision to describe something interesting in the first place.
Photography mostly eliminated the once-indispensable portrait artist, among other formerly-dependable lines of work.
There's a line to be drawn somewhere between artist and craftsperson. Creating beautiful things to a brief has always been a teachable skill, and now we're teaching it to machines. And, we've long sought to mass-produce beautiful things anyway. Think textiles, pottery, printmaking, architectural adornments.
Can AI replace an artist? Or is it just a new tool that can be used, as photography was, for either efficiency _or_ novel artistic expression?
> until the day it starts teaching artistry to its users.
It almost definitely can start teaching artistry to its users, and the same people who are mad in this thread will be mad that it's taking away jobs from art instructors.
The central problem is the same and it's what Marshall Brain predicted: If AI ushers in a world without scarcity of labor of all kinds, we're going to have to find a fundamentally new paradigm to allocate resources in a reasonably fair way because otherwise the world will just be like 6 billionaire tech executives, Donald Trump, and 8 billion impoverished unemployed paupers.
And no, "just stop doing AI" isn't an option, any more than "stop having nuclear weapons exist" was. Either we solve the problems, or a less scrupulous actor will be the only ones with the powerful AI, and they'll deploy it against us.
The first two paragraphs of your argument could be used to discuss whether Photography (Camera is doing most of the work) or Digital Drawing (Photoshop is doing most of the work) are art.
The work a camera does is capturing the image in front of the photographer. "Art" in the context of photography is the choice of what in the image should be in focus, the angle of the shot, the lighting. The camera just captures that; it doesn't create anything that isn't already there. So, not even remotely the same thing as AI Gen.
The work of Krita/Inkscape/etc (and technically even Photoshop) is to convert the artistic strokes into a digital version of how those strokes would appear if painted on a real medium using a real tool. It doesn't create anything that the artist isn't deliberately creating. So, not even remotely the same thing as AI Gen.
AI Gen, as demonstrated in the linked page and in the tool comparison, is doing all of the work of generating the image. The only work a human does is to select which of the generated images they like the best, which is not a creative act.
In your example, the "come up with your own story" part is the creative part. But you're not "directing" the AI to generate it for you. You're just giving it a command. You're selecting from the results it outputs, but you're not controlling the output.
A film director is a creative. Ultimately, they are in charge of "visualizing" a screenplay": the setting, the the design of the set or the utilization of real locations, the staging of the actors within a scene, the "direction" of the actors (i.e., how they should act out dialog or a scene, lighting, the cinematography, the use of stunts, staging shots to accommodate the use of VFX, the editing (meaning, the actual footage that comprises the movie).
There's an old show on HBO, Project Greenlight, that demonstrates what a director does. They give 2 directors the same screenplay and budget and they make competing movies. The competing movies are always completely different...even though they scripts are the same. (In the most extreme example from one of the later seasons, one of the movies was a teen grossout comedy, and the competing movie was some sort of adult melodrama.)
So 1. being able to bring your own story come to life automatically is cool in itself, and would result in a lot of creative media that is not possible now. Do you know how many people have their own stories, plays, etc that are dying to find someone rich enough to get them published?
2. Using AI can be can be an iterative process. Generate this scene, make this look like that, make it brighter colors, remove this, add this, etc. That's all carefully crafting the output. Now generate this second scene, make the transition this way, etc. I don't see how that's at all different from a director giving their commands to workers, except now you actually have more creative control (given AI gets good enough)
1. We already have that now, it's called Word. Most people are just too lazy to write out their story. AI doesn't improve the situation, it makes it worse. It will become vastly harder to find the good stuff in the avalanche of crap.
2. Current AI can't do what you're describing, so the biggest difference is that you're posing a hypothetical against the real world. But more specifically: the director already has a specific vision in their hand; the purpose of the "direction" is to bring this vision into reality within the scope of their budget and resources. With AI, you have a general idea and the AI creates its own vision and you pick what you like the best, until you ultimately realize the AI isn't going to get what you actually want and you settle for the best the AI can do for you. So, completely different.
>But you're not "directing" the AI to generate it for you. You're just giving it a command.
That's what direction is though. Film directors prompt their actors and choose the results they like best (among many other commands to many other groups)
>You're selecting from the results it outputs, but you're not controlling the output.
The prompt controls the output (and I bet you'd have more control over the AI than you'd have over a drunk Marlon Brando)
Not even if you are directing and refining it? What if i smudge out sections repeatedly and over the course of say 20 iterations produce a unique image that matches closely what i am imagining, and that has not be seem before?
Buñuel would disagree with you: "The peak of film-making will be reached when you are able to take a pill, switch off the lights, sit facing a blank wall and project on it, directly from your eyes, the film that passes through your head."
if I wasnt around to witness it over the last 10 years I would have thought most commenters on HN were bots pretending to be offended and gatekeeping for obscure profit motives.
So the bad news is people are just insecure, jealous, pedantic, easy to offend, highly autistic - and these are the smart ones.
The good news, is with dead internet theory they will all be replaced with bots that will atleast be more compelling make some sort of sense.
"Oh you're posting on hackernews, if you don't suck google's dick and every single gadgets megacorps shit out you must be highly autistic".... interesting take
If you call motives of designers, painters, musicians, filmmakers and other c-word professionals obscure, then I have bad news for whatever occupation you have for a living. Maybe it's just because it depends on how well AI sells, or because there's suddenly a perceived window of opportunity wide open at some fee for everyone unable to c-word anything on their own to finally become able, even if by relying on corporate crutches, the window you deem endangered by said gatekeepers?
Individuals won’t be able to do anything. The artist here is the LLM. There is no AI art where the human in the loop carries any significance. Proof of that is you can’t replicate their work using tbt same LLM. In AI art, the AI is the artist. The human is just a client making a request.
And who owns the AI?
It’s delusional. Stop falling for the mental jiu Jitsu from the large AI labs. You are not becoming an artist by using a machine to make art for you. The machine is the artist. And you don’t own it.
It's funny that you're defending creativity by being close-minded about a creative new way to explore it. You're being your judgment of an entire new medium based on a few early examples. It's as if you're downplaying photography just based on the few first blurry, dark clichés produced. Let's keep our minds open to new forms of creativity if we really care about it so much.
Rap is really the best example of how stupid these discussions are.
The language models are amazing at rhyming so by the logic then anyone can become a rapper and that is going to put current rappers out of business.
Only someone who has never tried to rap could possibly believe this. Same thing with images, same thing with music, same thing with literally everything involving human creativity.
It does but so does it encourage efortlessness and haste. It definitely is something new, it remains to be see whether creatives bring it to new heigts as with other mediums. I remain a bit skeptical but am open to it. One thing is certain, a tsunami of content is upon us.
> burying the creatives among us in a pile of AI generated content.
Isn't the creativity in what you put in the prompt? Isn't spending hundreds of hours manually creating and rigging models based on existing sketch the non-creative work that is being automated here?
How does a prompt describe creativity? It's a vision so far off that it's so frustrating because greater creativity came from limited tools, greater creativity came from imperfections, a different point of view, love, a slightly off touch of a painter or a guitar player, the wood of the instrument and the humidity affecting. I can go on and on, prompts are a reduction to the minimum term of everything you'd want to describe, no matter how much you can express via a prompt
I agree that limitation does lead to some creativity, but I wouldn't say lack of limitation means no creativity. Saying prompts have no creativity is like saying books or scripts have no creativity compared to a movie. Not only that, these tools can actually take images and sketches.
Imagine a world where you have a scene fully sketched out in your head (i.e. creativity), you have the script of what will happen, sketches of what the scene looks light, visual style, etc. You want to make that become reality. You could spend a ton of time and money, or you could describe it and provide sketches to an AI to make it come true.
Yes, the limitations in the former can make you take creative shortcuts that could themselves be interesting, but the latter could be just as true to your original vision.
Imagine a world where(spoiler alert you shouldn't it's already there) your vision is limited by yourself and a light technician gives a different point of view that you like, a punk pioneer in your team that is the computer graphics chief animator that gives life to your animatronics(jurassic park, and still unmatched today, cgi looks so fake and cheap). And yes you should spend a lot of time, not necessarily money, because art is time and something you have spent zero time in it its valued zero money
No it's not because you aren't creating the painting, the AI is. Having an idea is not sufficient to create art, you have to sit down and actually create it. If you hired a painter to create the work based off your poem, I'm not going to attribute that painters creation to you.
So deliberately writing a prompt that meticulously describes how a generated photo would look like isn't creative, but pushing a button for a machine to take the photo for you is??!! If anything, it's the way around!
Of course that's not what I believe, but let's not limit the definition of what creativity based on historical limitations. Let's see what the new generation of artists and creators will use this new capability to mesmerize us!
No, it isn’t, because the prompt doesn’t have a millionth of the information density of the output.
Merely changing a seed number will provide endless different outputs from the same single prompt from the same model; rng.nextInt() deserves as much artist credit as the prompter.
Your meticulous prompt is using the work of thousands of experts, and generating a mashup of what they did/their work/their commitment/their livelihood.
Their placement of books. Their aesthetic. The collection of cool things to put into a scene to make it interesting. The lighting. Not yours. Not from you/not from the AI. None of it is yours/you/new/from the AI. It's ALL based underneath on someone else's work, someone else's life, someone else's heart and soul, and you are just taking it and saying 'look what I made'. The equivalent of a 4 year old being potty trained saying 'look I made a poop'. We celebrate it as a first step, not as the friggen end goal. The end goal is you making something uniquely you, based on your life experience, not on Bob the prop guys and Betty the set designer whose work/style you stole and didn't even have the decency to reference/thank.
And your prompt won't ever change dramatically, because there isn't going to be much new truly creative seedcorn for AI to digest. Entertainment will literally go into limbo/Groundhog Day, just the same generative, derivative things/asthetics from the same AI dataset.
And that's exactly how your brain work. What you call "creativity" is nothing more than exactly that: mixing ideas and thoughts you were exposed to. We're all building on others' work. The only difference is that computers do it on a much larger scale. But it's the very same process.
This is completely absurd and reductive point of view, which I always assume is a cop out. Just because it's called "machine learning" doesn't mean it actually has anything to do with how human learning or human brain works, and it's certainly not "exactly how" or "very same". There's much more going on on in human creative process, aside from mere "mixing": personal experience, understanding of the creative process, technique and style development, subtext, hidden ideas and nuances, etc. Computers are very good at mixing and combining, but this is not even close to what goes into actual creative process. I hate this argument
> personal experience, understanding of the creative process, technique and style development, subtext, hidden ideas and nuances
All of these are just human being exposed more to life and learning new skills, in other words -- having more data. LLM already learns those skills and encounters endless experience of people in its training data.
> I hate this argument
That's very subjective. You don't know how the brain works.
You seem to not fully understand the quote. LLMs learn patterns/noises from an existing output, these are not skills nor endless experience learned. It's like saying you learn how to make a cake by learning how it should look like not how it's composed. LLMs mock the studio ghibli style images, didn't invent the style or learned the endless experience the studio accumulated over the years. In fact it's a mocking of the images and it just looks horrible
It's not just more data, it's deeper understanding of the fundamentals, of the idea and of the tools used, as well as the process of creation itself. It's what makes studying art interesting: why did author chose to do this and that, what's their style, what was the process, etc. For LLM the answers will universally be "because it was in the prompt and there was appropriate training data" and "the author prompted the model until the model returned something tolerable". You may argue that not all art has or needs depth, or that not all people are interested in it, but that doesn't mean that we should fill our cultures with empty boring slop.
> That's very subjective
I was expressing my opinion of this argument which absolutely is subjective
> You don't know how the brain works.
Neither does grandparent comment's author, didn't stop them from making much bolder claims.
But you aren't being creative here. Just using the 'average' of tons of actually creative peoples work to create an 'average' computer predicted scene. The opposite of art. Warhol already did it and did it better.
If I see a painting, I see an interpretation that makes me think through someone else's interpretation.
If I see a photograph, I don't analyze as much, but I see a time and place. What is the photographer trying to get me to see?
If I see AI, I see a machine dithered averaging that is/means/represents/construes nothing but a computer predicted average. I might as well generate a UUID, I would get more novelty. No backstory, because items in the scene just happened to be averaged in. No style, just a machine dithered blend. It represents nothing no matter the prompt you use because the majority is still just machine averaged/dithered non-meaning. Not placed with intention, focused with real vision, no obvious exclusions with intention. Just exactly what software thinks is the most average for the scene it had described to it. The better AI gets, the more average it becomes, and the less people will care about 'perfectly average' images.
It won't even work for ads for long. Ads will become wild/novel/distinct/wacky/violations of AI rules/processes/techniques to escape and belittle AI. To mock AI. Technically perfect images will soon be considered worthless AI trash. If for no other reason than artists will only be rewarded for moving in directions AI can't going forward. The second Google/OpenAI reach their goal, the goal posts will move because no one wants procedural/perfectly average slop.
artist have a style,you can see a work of art and know who made it, with these AI images its all random all over the place no direction, they can call them self's artists but i will never see them as that
Eh, every AI "artist" want the cachet of being an artist without any of the effort, but they're competing with other AI "artists" so they have no choice but to unleash a firehose of content sludge onto the commons in a race to the bottom.
For better or worse, a big chunk (if not most) of the AI development probably does go into non-creative work like matching ads against users and ranking search results
It's just not what gets the exciting headlines and showcases
Distribution of art (particularly digital) is a recent phenomenon. Prior to that, art in human history was one-off. Are we just going back to that time?
Similarly with music, prior to recording tech, live performance was where it was at.
You could look at the digital era as a weird blip in art history.
It's definitely coming. Creative work is first because there's zero constraints on it. Doing non-creative work, you're bound to hit a constraint - real world or otherwise - immediately, and AI is only just starting to navigate that.
The fact that so many feel the same way about this technology (I do too!) is an indictment of humanity, not the technology itself.
We _could_ use this to empower humans, but many of us instinctively know that it will instead be used to crush the human spirit. The end result of this isn’t going to be an expansion of creative ability, it’s going to be the destruction of creative jobs and the capture of these creative mediums by a few large companies.
> The end result of this isn’t going to be an expansion of creative ability, it’s going to be the destruction of creative jobs and the capture of these creative mediums by a few large companies.
I agree , but that's the negative. The positive will be that almost any service you can imagine (medical diagnosis, tax preparation, higher education) will come down to zero, and with a lag of perhaps a decade or two it will meet us in the physical world with robo-technicians, surgeons and plumbers. The cost of building a new house or railway will plummet to the cost of the material and the land, and will be finished in 1/10 of the time it takes today.
The main problem to me is that there's a lag between the negatives and the positives. We're starting out with the negatives and the benefits may take a decade or two to reach us all equally.
> The positive will be that almost any service you can imagine (medical diagnosis, tax preparation, higher education) will come down to zero
Why would you want massive amounts more of those things? In fact I might even argue that medicine, taxation and education are a net negative on society already. And that to the extent that there seems to be scarcity, it's mainly a distribution problem having to do with entrenched interests and bureaucracy.
> The cost of building a new house or railway will plummet to the cost of the material and the land
> Why would you want massive amounts more of those things? In fact I might even argue that medicine, taxation and education are a net negative on society already. And that to the extent that there seems to be scarcity, it's mainly a distribution problem having to do with entrenched interests and bureaucracy.
I'm not sure what you mean. In my country getting a specialist to take a look at you can take weeks, the scarcity is that there's not enough doctors. For sure many people get delayed and suboptimal diagnosis (even if you finally get to see the specialist, he may have 10 mintues for you and 50 other patients to see that day). A.I can simply solve this.
> The cost of building a new house or railway will plummet to the cost of the material and the land
>> That's is the actual scarcity tho
Not necessarily, the labor costs a tremendous amount, and also it might be that we don't need to cram tens of millions of people around cities anymore if most work is automated, we can start spreading out (again, this will take decades and I'm not denying we have pressing problems in the immediate future).
So the humankind was waiting for AI to bring down all costs to zero, good lord! I thought it was waiting for steam engine, penicillin, railroads, aviation, robotics, computers, nuclear energy, space flight to bring that forth!
> We _could_ use this to empower humans, but many of us instinctively know that it will instead be used to crush the human spirit. The end result of this isn’t going to be an expansion of creative ability, it’s going to be the destruction of creative jobs and the capture of these creative mediums by a few large companies.
The kind of argument which boils down to "the death of the human spirit is imminent, because it is never OK to stop where we are, and only a step forward is possible, because there are plenty steps already taken behind"
you act as if the human population has no agency to choose what they want? This will be another tool for good and bad. People will make beautiful things the world hasn't seen before, and others will use it for propaganda. just like all things we touch
Expressing themselves by generating boilerplate content?
Creativity is a conversation with yourself and God. Stripping away the struggle that comes with creativity defeats the entire purpose. Making it easier to make content is good for capital, but no one will ever get fulfillment out of prompting an AI and settling with the result.
Check out all the creatives on /r/screenwriting, half the time they are trying to figure out how to "make connections" just to get a story considered. It's a fucking nightmare out there. Whatever god is providing us with AI is the greatest gift I could imagine to a creative.
AI could be useful if used like any other tool, but not as an all in box where everything is done for you minus the prompt. Im actually worried people will become lazy
People will always become lazy with new tools. But not all people.
It'll lower the barrier of entry (and therefore the quality floor before people feel comfortable sharing something "they made" if they can deflect with an easy "the AI made this" versus "I put XY0 hours into this"), but it'll also empower people who wouldn't otherwise even try to create new things and, presumably, follow their passion to learn and do more.
Creativity is an expression. It comes from the heart. Hard work isn't always the greatest vehicle for creativity. We just think it is. I've seen plenty of things that clearly took a lot of execution but fundamentally lack creativity, often becoming an exhibition in technical virtuosity.
Here's something you can try to prove it to yourself. Sit down and write a novel. It'll be like squeezing blood out of a rock unless your heart is ready to do it freely. You'll see that if you force yourself through hard work to do it, you'll just end up with something that people will laud as creative due to the execution but it'll lack everything about free-flowing creativity. Good programmers are lazy, so are good creatives, but now I'm just repeating myself.
It's a lot easier squeezing blood out of a heart, especially for the lazy.
exactly, creatives and everyone else can always do something fulfilling for themselves just like before AI. They can struggle all they want and continue doing it for no capital. because that process is fulfilling to them.
> Yes but how many will sign up for that? Im sure few will continue to do so but crea
It’s not important to me that they do.
> Im sure few will continue to do so but creativity will certainly take a big hit.
I’ve seen the workflows for AI generated films like https://youtu.be/x6aERZWaarM?si=J2VHYAHLL3og32Ix and I find it to be very creative. Its more interesting to me that this person would never have raised capital and tried to direct this, but this is much closer to what they wanted to create. I’m also entertained by it, whether I was judging it for generative AI issues or not.
the level of discipline needed in a trade has gone down in almost every trade, including all mediums of art, for centuries
its not really anyone's problem, and generally limited to the people that made way too much of their identity to be based on a single field, that they feel they have to gatekeep it
its great that people can express themselves closer to their vision now
Recordings of who? Not only sad but a disaster, I'm sorry but anyone that ever tried to play an instrument seriously knows how much human touch/imperfections come into play, otherwise you're just an anonymous guy playing in a cover band(like the ai will do)
A tsunami of effortless content is upon us and that will change many things including tastes, probably for the worse. People not having to learn instruments because same can be done with a prompt is a tragic loss for humanity, not because human work is better but because of the lost experience and joy of learning, connection with the self and others and so many other things.
> because of the lost experience and joy of learning, connection with the self and others and so many other things.
Nothing prevents human from continue doing just that, precisely because it brings joy and satisfaction. Painting, photography classes are still popular, if not more, in the age of digital photography.
Robotics will come in the next few years. If you believe the AI2027 guys, though, the majority of work will be automated in the next 10 years, which seems more and more plausible to me every day.
Are you independently wealthy enough to benefit from that or someone who should invest in suicide pills for themselves and their family if that day comes?
Why invest in weaksauce suicide pills when you could instead invest in nitrogen compounds and suicide bomb the tallest nearby building? Just because you've already lost doesn't mean they get to win, let alone survive.
Some people haven't overcome their childhood desire to die or suffer so as to make the parents regret their decision not to buy that candy or take the puppy home. They imagine dismal future as a glorified way to suffer. Talk about cyberpunk - there's that sweet alluring promise to spend a whole life eating instant ramen sitting next to a window with a blinking neon sign and endless rain behind it, coding routinely to a lofi soundtrack, or lurking lonesomely about the techno-slum neighbourhood hiding their faces from CCTV behind masks
There’s less talk about automating non-creative work because it’s not flashy. But I can promise it’s a ton of fun, and you can co-design these automations with an LLM.
Multimodal LLMs are currently the natural research step towards AGI robots that can do mundane non-creative work. I believe this is just the reality of the situation. If you can generate a video of a robot doing the dishes then your model understands the physical world quite well. That should be useful for robot control.
Making a movie is not accessible to most people and it's EVERYONES dream. This is not even there yet, but I have a few movies I need to make and I will never get a cast together and go do it before I die. If some creatives need to take a backseat so a million more creatives can get a chance, then so be it.
Nobody will watch anyone's "creations". The TV or whatever device watch on will observe what you and everyone else engage with and how you interact with it and create content for you on the fly, just like Instagram and TikTok's feeds do now.
AI creatives can enjoy the brief blip in time where they might get someone else to watch what they've created before their skills become obsolete in an exponentially faster rate just like everyone else's.
Then everyone can just get their own personal movies and infinite content stream. Honestly people would probably like that given how atomized society has become.
What is non-creative work? I think the term reeks of elitism. Every job is creative, even picking up garbage can become an art when one puts effort in it.
There is a more sensical distinction between work that is informational in nature, and work that is physical and requires heavy tools in hard-to-reach places. That's hard to do for big tech, because making tests with heavy machinery is hard and time consuming
It makes me sad that the US and western Europe which have been the most flexible and forward-thinking societies in the world for generations have now memed themselves into fretting and hand-wringing about technical advances that are really awesome. And for what? The belief that illustration and filmmaking which have always been hobbies for the vast majority of participants should be some kind of jobs program?
People aren't looking forward to companies playing the "how much sawdust can you put in a rice crispy before people notice the difference" experiment on the entertainment industry. The quality of acting, scripting, lighting, and animation in the film/television industry already feels second rate to stuff being made before 2020. The cost cutting and gutting of cultural products is becoming ridiculous, and this technology will only be an accelerant.
reply