Skip to main content
Exploring the Images Used to Train Stable Diffusion’s Image Generator

Exploring the Images Used to Train Stable Diffusion’s Image Generator

waxy.orgwaxy.org3 weeks ago in#Web Design Love39

One of the biggest frustrations of text-to-image generation AI models is that they feel like a black box. We know they were trained on images pulled from the web, but which ones? As an artist or photographer, an obvious question is whether your work was used to train the AI model, but this is surprisingly hard to answer. Sometimes, the data isn’t available at all: OpenAI has said it’s trained DALL-E 2 on hundreds of millions of captioned images, but hasn’t released the proprietary data. By contrast, the team behind Stable Diffusion have been very transparent about how their model is trained. Since it was released publicly last week, Stable Diffusion has exploded in popularity, in large part because of its free and permissive licensing, already incorporated into the new Midjourney beta, NightCafe, and Stability AI’s own DreamStudio app, as well as for use on your own computer. But Stable Diffusion’s training datasets are impossible for most people to download, let alone search, with metadata for millions (or billions!) of images stored in obscure file formats in large multipart archives. So, with the help of my friend Simon Willison, we grabbed the data for over 12 million images used to train Stable Diffusion, and used his Datasette project to make a data browser for you to explore and search it yourself. Note that this is only a small subset of the total training data: about 2% of the 600 million images used to train the most recent three checkpoints, and only 0.5% of the 2.3 billion images that it was first trained on. Screenshot of the LAION-Aesthetic data browser in Datasette Go try it right now at laion-aesthetic.datasette.io! Read on to learn about how this dataset was collected, the websites it most frequently pulled images from, and the artists, famous faces, and fictional characters most frequently found in the data. Data Source Stable Diffusion was trained off three massive datasets collected by LAION, a nonprofit whose compute time was largely funded by Stable Diffusion’s owner, Stability AI. All of LAION’s image datasets are built off of Common Crawl, a nonprofit that scrapes billions of webpages monthly and releases them as massive datasets. LAION collected all HTML image tags that had alt-text attributes, classified the resulting 5 billion image-pairs based on their language, and then filtered the results into separate datasets using their resolution, a predicted likelihood of having a watermark, and their predicted “aesthetic” score (i.e. subjective visual quality). Collage of some of the images with the highest “aesthetic” score, largely watercolor landscapes and portraits of women Stable Diffusion’s initial training was on low-resolution 256×256 images from LAION-2B-EN, a set of 2.3 billion English-captioned images from LAION-5B‘s full collection of 5.85 billion image-text pairs, as well as LAION-High-Resolution, another subset of LAION-5B with 170 million images greater than 1024×1024 resolution (downsampled to 512×512). Its last three checkpoints were on LAION-Aesthetics v2 5+, a 600 million image subset of LAION-2B-EN with a predicted aesthetics score of 5 or higher, with low-resolution and likely watermarked images filtered out. For our data explorer, we originally wanted to show the full dataset, but it’s a challenge to host a 600 million record database in an affordable,  » Read More

Like to keep reading?

This article first appeared on waxy.org. If you'd like to continue this story, follow the white rabbit.

View Full Article
Laravel Vs Symfony: Answering All The Questions To Make a Better Choice

Laravel Vs Symfony: Answering All The Questions To Make a Better Choice

#Dev
How to Channel a Daily Vision into a 20-Year Photography Career

How to Channel a Daily Vision into a 20-Year Photography Career

#Silicon Valley
WordPress Punts Locally Hosted Fonts for Legacy Default Themes to 6.2 Release

WordPress Punts Locally Hosted Fonts for Legacy Default Themes to 6.2 Release

#Web Design
Fresh For Designers

Is the Dynamic Island plain stupid or the next revolutionary UX pattern?

#All

Let's talk about Web Design

The term "web design" describes the layout of websites that are seen online. Instead of software development, it typically refers to the user experience components of website development. The primary focus of web design used to be creating websites for desktop browsers, but from the middle of the 2010s, designing for mobile and tablet browsers has gained significance.

What is a webdesigner?

A web designer is responsible for a website's look, feel, and occasionally even content. For instance, appearance refers to the colors, text, and images utilized. Information's organization and categorization are referred to as its layout. An effective web design is user-friendly, aesthetically pleasing, and appropriate for the target audience and brand of the website. Many websites focus on keeping things simple so that viewers won't be distracted or confused by additional information and functionality. Removing as many potential sources of user annoyance as possible is a crucial factor to take into account because the foundation of a web designer's output is a site that gains and nurtures the trust of the target audience.

Responsive and adaptive design are two of the most popular techniques for creating websites that function well on both desktop and mobile devices. In adaptive design, the website content is fixed in layout sizes that correspond to typical screen sizes, while in responsive design, information moves dynamically based on screen size. A layout that is as consistent as possible across devices is essential to preserving user engagement and trust. Designers must be cautious when giving up control of how their work will appear because responsive design can be challenging in this area. While they might need to diversify their skill set if they are also in charge of the content, they will benefit from having complete control over the final output.

What does a web design worker do?

A web designer is a member of the IT industry who is in charge of planning a website's structure, aesthetic appeal, and usability.

A skilled site designer must possess both technical know-how and creative graphic design abilities. They must be able to envision how a website will seem (its graphical design) and how it will operate (conversion of a design into a working website).

The terms web developer and designer are frequently used interchangeably but erroneously. In order to construct more complex interactions on a website, such as the integration with a database system, a web developer is frequently more likely to be a software developer who works with programming languages.