1/20th of civitai user prompts added as 300Mb safetensor file to the CLIP interrogator
https://lemmy.world/post/20812652
1/20th of civitai user prompts added as 300Mb safetensor file to the CLIP interrogator - Lemmy.World
Image shows list of prompt items before/after running ‘remove duplicates’ from a
subset of the Adam Codd huggingface repo of civitai prompts:
https://huggingface.co/datasets/AdamCodd/Civitai-2m-prompts/tree/main
[https://huggingface.co/datasets/AdamCodd/Civitai-2m-prompts/tree/main] Link to
notebook here:
https://huggingface.co/datasets/codeShare/fusion-t2i-generator-data/blob/main/Google
Colab Jupyter Notebooks/fusion_t2i_CLIP_interrogator.ipynb
[https://huggingface.co/datasets/codeShare/fusion-t2i-generator-data/blob/main/Google%20Colab%20Jupyter%20Notebooks/fusion_t2i_CLIP_interrogator.ipynb]
//—// Removing duplicates from civitai prompts results in a 90% reduction of
items! From 4.8 million-> 0.417 million items. If you wish to search this set ,
you can use the notebook above. Unlike the typical pharmapsychotic CLIP
interrogator , I pre-encode the text corpus ahead of time. Additionally , I’m
using quantization on the text corpus to store the encodings as unsigned
integers (torch.uint8) instead of float32 , using this formula:
[https://lemmy.world/pictrs/image/6c03bb16-66f1-458e-8124-f68ec8ef1f01.png] For
the clip encodings , I use scale 0.0043. The TLDR is that you divide the float32
value with 0.043 , round it up to the closest integer , and then add the
zero_point until all values within the encoding is above 0. A typical zero_point
value for a given encoding can be 0 , 30 , 120 or 250-ish. In summary , a pretty
useful setup for me when I need prompts for stuff.
//—// I also have a 1.6 million item fanfiction set of tags loaded from
https://archiveofourown.org/ [https://archiveofourown.org/] Its mostly character
names. They are listed as fanfic1 and fanfic2 respectively. //—// Upcoming plans
is to include a visual representation of the text_encodings as colored cells
within a 16x16 grid. A color is an RGB value (3 integer values) within a given
range , and 3 x 16 x 16 = 768 , which happens to be the dimension of the CLIP
encoding //—// Thats all for this update.
This is the perchance emoji ⚄ - Lemmy.World
Link : https://www.htmlsymbols.xyz/unicode/U+2684
[https://www.htmlsymbols.xyz/unicode/U+2684] The emoji used on the perchance
website. Might be useful to someone.
Quem 2.5 (State-of-art chatbot) was just released
https://lemmy.world/post/19944442
Quem 2.5 (State-of-art chatbot) was just released - Lemmy.World
Link : https://huggingface.co/spaces/Qwen/Qwen2.5
[https://huggingface.co/spaces/Qwen/Qwen2.5] Background (posted today!) :
https://qwenlm.github.io/blog/qwen2.5-llm/
[https://qwenlm.github.io/blog/qwen2.5-llm/] //----// These were released today.
I have 0% knowledge what this thing can do, other than it seems be a really good
LLM.
New dedicated repo for my notebooks
https://lemmy.world/post/19925257
New dedicated repo for my notebooks - Lemmy.World
Link :
https://huggingface.co/datasets/codeShare/text-to-image-prompts/tree/main/Google
Colab Notebooks
[https://huggingface.co/datasets/codeShare/text-to-image-prompts/tree/main/Google%20Colab%20Notebooks]
Still have a ton of data I need to process and upload.
[Dev Diary] More sets added to the NND CLIP interrogator
https://lemmy.world/post/19878430
[Dev Diary] More sets added to the NND CLIP interrogator - Lemmy.World
This post is a developer diary , kind of. I’m making an improved CLIP
interrogator using nearest-neighbor decoding. It doesn’t require GPU to run, and
is super quick. The reason for this is that the text_encodings are calculated
ahead of time , unlike the Pharmapsychotic model aka the “vanilla” CLIP
interrogator :
https://huggingface.co/spaces/pharmapsychotic/CLIP-Interrogator/discussions
[https://huggingface.co/spaces/pharmapsychotic/CLIP-Interrogator/discussions]
//----// This post gonna be a bit haphazard, but that’s the way things are
before I get the huggingface gradio module up and running.
Then it can be a fancy “feature” post , but no clue when I will be able to code
that. So better to give an update on the ad-hoc solution I have now. The NND
method I’m using is described here , in this paper which presents various ways
to improve CLIP Interrogators: https://arxiv.org/pdf/2303.03032
[https://arxiv.org/pdf/2303.03032]
[https://lemmy.world/pictrs/image/d499215c-248d-43b5-9c8c-8360581229da.png]
Easier to just use the notebook then follow this gibberish. We pre-encode a
bunch of prompt items , then select the most similiar one using dot product.
Thats the TLDR. Right now the resources available are the ones you see in the
image. I’ll try to showcase it at some point. But really , I’m mostly building
this tool because it is very convenient for myself + a fun challenge to use
CLIP.
It’s more complicated than the regular CLIP interrogator , but we get a whole
bunch of items to select from , and can select exactly “how similiar” we want it
to be to the target image/text encoding. The {itemA|itemB|itemC} format is used
as this will select an item at random when used on the perchance text-to-image
servers, in in which I have a generator where I’m using the full dataset ,
https://perchance.org/fusion-ai-image-generator
[https://perchance.org/fusion-ai-image-generator] It takes minutes to load a
fraction of the sets from perchance servers before this generator is “up and
running” so-to speak. I plan to migrate the database to a Huggingface repo to
solve this. https://huggingface.co/datasets/codeShare/text-to-image-prompts
[https://huggingface.co/datasets/codeShare/text-to-image-prompts] The
{itemA|itemB|itemC} format is also a build-in random selection feature on
ComfyUI , coincidentally :
[https://lemmy.world/pictrs/image/acdd7ee0-7a4e-425b-b0ec-56ffc85e73ab.png]
Source :
https://blenderneko.github.io/ComfyUI-docs/Interface/Textprompts/#up-and-down-weighting
[https://blenderneko.github.io/ComfyUI-docs/Interface/Textprompts/#up-and-down-weighting]
Links/Resources posted here might be useful to someone in the meantime.
[https://lemmy.world/pictrs/image/9230bde6-42a7-4f6e-881b-0af25fec4458.png] You
can find tons of strange modules on the Huggingface page :
https://huggingface.co/spaces [https://huggingface.co/spaces]
[https://lemmy.world/pictrs/image/00fa2b8c-df52-4188-8236-a1a470939a81.png] For
now you will have to make do with the NND CLIP Interrogator notebook :
https://huggingface.co/codeShare/JupyterNotebooks/blob/main/sd_token_similarity_calculator.ipynb
[https://huggingface.co/codeShare/JupyterNotebooks/blob/main/sd_token_similarity_calculator.ipynb]
[https://lemmy.world/pictrs/image/1a19f5c4-2484-48df-b584-18f55d657d44.png]
text_encoding_converter (also in the NND notebook) :
https://huggingface.co/codeShare/JupyterNotebooks/blob/main/indexed_text_encoding_converter.ipynb
[https://huggingface.co/codeShare/JupyterNotebooks/blob/main/indexed_text_encoding_converter.ipynb]
I’m using this to batch process JSON files into json + text_encoding paired
files. Really useful (for me at least) when building the interrogator. Runs on
the either Colab GPU or on Kaggle for added speed: https://www.kaggle.com/
[https://www.kaggle.com/] Here is the dataset folder
https://huggingface.co/datasets/codeShare/text-to-image-prompts:
[https://huggingface.co/datasets/codeShare/text-to-image-prompts:]
[https://lemmy.world/pictrs/image/388bbdbd-8277-49c0-9e97-82534cd938c0.png]
Inside these folders you can see the auto-generated safetensor + json pairings
in the “text” and “text_encodings” folders.
The JSON file(s) of prompt items from which these were processed are in the
“raw” folder.
[https://lemmy.world/pictrs/image/68775dfb-d674-4673-a20b-78acd52ff593.png] The
text_encodings are stored as safetensors. These all represent 100K female first
names , with 1K items in each file. By splitting the files this way , it uses
way less RAM / VRAM as lists of 1K can be processed one at a time.
[https://lemmy.world/pictrs/image/bfe821f0-4015-4c54-89b1-659262984991.png]
//-----// Had some issues earlier with IDs not matching their embeddings but
that should be resolved with this new established method I’m using. The hardest
part is always getting the infrastructure in place. I can process roughly 50K
text encodings in about the time it takes to write this post (currently
processing a set of 100K female firstnames into text encodings for the NND CLIP
interrogator. ) EDIT : Here is the output uploaded
https://huggingface.co/datasets/codeShare/text-to-image-prompts/tree/main/names/firstnames
[https://huggingface.co/datasets/codeShare/text-to-image-prompts/tree/main/names/firstnames]
I’ve updated the notebook to include a similarity search for ~100K female
firstnames , 100K lastnames and a randomized 36K mix of female firstnames +
lastnames Sources for firstnames :
https://huggingface.co/datasets/jbrazzy/baby_names
[https://huggingface.co/datasets/jbrazzy/baby_names] List of most popular names
given to people in the US by year Sources for lastnames :
https://github.com/Debdut/names.io [https://github.com/Debdut/names.io] An
international list of all firstnames + lastnames in existance, pretty much .
Kinda borked as it is biased towards non-western names. Haven’t been able to
filter this by nationality unfortunately. //------// Its a JSON + safetensor
pairing with 1K items in each. Inside the JSON is the name of the .safetensor
files which it corresponds to. This system is super quick :)! I plan on running
a list of celebrities against the randomized list for firstnames + lastnames in
order to create a list of fake “celebrities” that only exist in Stable Diffusion
latent space. An “ethical” celebrity list, if you can call it that which have
similiar text-encodings to real people but are not actually real names. I have
plans on making the NND image interrogator a public resource on Huggingface
later down the line, using these sets. Will likely use the repo for perchance
imports as well: https://huggingface.co/datasets/codeShare/text-to-image-prompts
[https://huggingface.co/datasets/codeShare/text-to-image-prompts]
[Resource] [T2i] indexed text_encoding converter.ipynb + sneak preview
https://lemmy.world/post/19875602
[Resource] [T2i] indexed text_encoding converter.ipynb + sneak preview - Lemmy.World
Link:
https://huggingface.co/codeShare/JupyterNotebooks/blob/main/indexed_text_encoding_converter.ipynb
[https://huggingface.co/codeShare/JupyterNotebooks/blob/main/indexed_text_encoding_converter.ipynb]
I’m using this to batch process JSON files into json + text_encoding paired
files. Really useful (for me at least) when building the interrogator.
[https://lemmy.world/pictrs/image/9230bde6-42a7-4f6e-881b-0af25fec4458.png] I
have plans on making the NND image interrogator a public resource on Huggingface
later down the line, using these sets. Will likely use the repo for perchance
imports as well: https://huggingface.co/datasets/codeShare/text-to-image-prompts
[https://huggingface.co/datasets/codeShare/text-to-image-prompts]
[T2i] [SD1.5] I prompted myself into the backrooms
https://lemmy.world/post/19812973
[T2i] [SD1.5] I prompted myself into the backrooms - Lemmy.World
[https://lemmy.world/pictrs/image/59b3cd5a-517d-458c-9716-116f8d1cbc31.jpeg]
[https://lemmy.world/pictrs/image/445acb79-02ca-4f96-acbb-4cdbb8ab324a.png]
[https://lemmy.world/pictrs/image/d61331b5-5560-4b70-95d9-4ffd74229323.jpeg]
[https://lemmy.world/pictrs/image/e2058f97-bb46-4d92-b856-034ce1f3d422.jpeg]
[https://lemmy.world/pictrs/image/229731e3-c13f-4b32-b4b5-458052487698.jpeg]
[https://lemmy.world/pictrs/image/9c73e629-7692-4598-8739-97f37947cde9.jpeg]
[https://lemmy.world/pictrs/image/d0f87aad-92c8-4881-aa4e-50c5f85a7f51.jpeg]
prompt: “[#FTSA# : red carpet in background by architecture Tuymans and
pani jaan antibody hopped bine users eternity archives :0.1]”
https://perchance.org/fusion-ai-image-generator
[https://perchance.org/fusion-ai-image-generator]
I get this error from dynamic imports. Ideas?
https://lemmy.world/post/19799205
I get this error from dynamic imports. Ideas? - Lemmy.World
Error disappears by updating any HTML element on the fusion gen page.
Source: https://perchance.org/fusion-ai-image-generator
[https://perchance.org/fusion-ai-image-generator] Dynamic imports plugin:
https://perchance.org/dynamic-import-plugin
[https://perchance.org/dynamic-import-plugin] I let name = localStorage.name
[http://localStorage.name] , if it exists , when running dynamicImports(name).
I didn’t have this error when I implemented the localStorage thingy.
So I suspected this to be connected to some new added feature for dynamic
Imports.
Ideas on solving this? Code when I select names for dynamic import upon start
(the error only occurs upon opening/reloading the page) : _generator
gen_danbooru fusion-t2i-danbooru-1 fusion-t2i-danbooru-2 fusion-t2i-danbooru-3
gen_lyrics fusion-t2i-lyrics-1 fusion-t2i-lyrics-2 ... _genKeys gen_danbooru
gen_lyrics ... // Initialize getStartingValue(type) =>
_genKeys.selectAll.forEach(function(_key) { document[_key] = 'fusion-t2i-empty';
if (localStorage.getItem(_key) && localStorage.getItem(_key) != '' &&
localStorage.getItem(_key) != 'fusion-t2i-empty') { document[_key] =
localStorage.getItem(_key); } else { document[_key] =
[_generator[_key].selectOne]; localStorage.setItem(_key, document[_key]); }; });
... dynamicImport(document.gen_danbooru, 'preload'); ... if (type ==
"danbooru"): return document.gen_danbooru; }; // End of getStartingValue(type)
... _folders danbooru = dynamicImport(document.gen_danbooru ||
getStartingValue("danbooru"))
[T2i] [Prompts] Huggingface repo created for fusion-gen prompts
https://lemmy.world/post/19757588
[T2i] [Prompts] Huggingface repo created for fusion-gen prompts - Lemmy.World
Link:
https://huggingface.co/datasets/codeShare/text-to-image-prompts/blob/main/README.md
[https://huggingface.co/datasets/codeShare/text-to-image-prompts/blob/main/README.md]
Will update with JSON + text_encoding pairs as I process the sub_generators
[T2i] [Prompts] The contents of the fusion-gen sub-generators can now be downloaded as JSON files
https://lemmy.world/post/19737378
[T2i] [Prompts] The contents of the fusion-gen sub-generators can now be downloaded as JSON files - Lemmy.World
I’m adding this feature because I plan to build a Huggingface JSON repo of the
contents of my sub-generators , which I plan to use for my image interrogator:
https://huggingface.co/codeShare/JupyterNotebooks/blob/main/sd_token_similarity_calculator.ipynb
[https://huggingface.co/codeShare/JupyterNotebooks/blob/main/sd_token_similarity_calculator.ipynb]
Example: https://perchance.org/fusion-t2i-prompt-features-5
[https://perchance.org/fusion-t2i-prompt-features-5] List of currently updated
generators can be found here:
