Welcome to my site!
Hey, i am IkariDev.
I love anime especially serial experiments lain type animes, they just have a kind of.. special feeling.
Upvote if you like lain!
Click me!
Send merging suggestion
This site is kinda crap atm, and i don't really know what to write here currently.
If you want to know what i am currently up to you can check out my blog.
About
Hey, i am IkariDev.
I am a developer from germany doing various things, such as:
- Machine learning [ Large Language Models, Stable diffusion, other minor stuff ]
- Watching anime
- Writing software/scripts
My main programming languages i use:
- C#
- Python
Programming languages i want to learn/i am learning:
- JavaScript [ + Css, + HTML ]
- C++
What i like:
- Anime
- Edgy stuff (I mean, look at my website lol)
- Open-Source
- Muh waifu
What i dislike:
- OpenAI
- People who don't read HuggingFace descriptions before asking questions
You can also check me out on these places:
Nothing much to say about me, but if you really want to know more about me you can check out my blog.
Contact
You can currently contact me on these services:
- Discord [ username: ikaridev ]
Noromaid model series
19.12.2023 - 22:51 [GMT+2] NeverSleep DiscordName comes from:
Noro = No_robots(dataset),
Maid = Minerva AI Dataset
MinervaAI Dataset referring to: Aesir
Noromaid 0.1.1
Yet another merge with undi, came out 16.11.2023, main part of it a dataset by MinervaAI named aesir, other part is a modified version of No_Robots.
Noromaid 0.2 [ EXPERIMENTAL ]
As seen in the title, this model is pretty experimental!
This time we used MergeMonster, with many bad_phrases. We also used another MinervaAI dataset.
The 13b was published on 16.12.2023, the 7b was released on 21.12.2023, further B's will be released when we get more feedback
Noromaid 0.3
Noromaid 0.3 has: new datasets, normal alpaca Instruct(with instruction, input, and response header) instead of our modified alpaca.
It is a complete retrain based on the same model as 0.1.1 and 0.1.
Noromaid 0.4
Noromaid 7b v0.4 is a fully finetuned 7b Mistral trained on multiple RP dataset, modified by our own hand, and redone entirely from Alpaca to Chatml. The switch from Alpaca to Chatml and the addition of a new dataset from Aesir made it even better than 0.3.
Noromaid 7b v0.4 DPO is the same as Noromaid 0.4 7b. Additionally, it was trained on top with 3 DPO datasets including the Intel DPO for reasoning, and some uncensoring.
Spin-Off's
(standalone)
Mixtral
- Noromaid-v0.1-mixtral-8x7b-v1
Noromaid-v0.1-mixtral-8x7b-v1 is a fine-tune of Mixtral 8x7b, trained on various RP datasets including LimaRP and Aesir, ToxicDPO without warning for decensoring and norobots, rewritten to use a modified Alpaca prompting to be on a par with ChatML or other conversational formats. This version contains 3x Alpaca modified datasets (for the RP one) and 2 shareGPT datasets. LimaRP token length, input, and output size got wiped out, and were separated into chunks of +8k context conversation (this is the maximum LimaRP can offer).
- Noromaid-v0.1-mixtral-8x7b-v2
Noromaid-v0.1-mixtral-8x7b-v2 is a fine-tune of Mixtral 8x7b, trained on various RP datasets including LimaRP and Aesir, ToxicDPO without warning for decensoring and norobots, rewritten to use a modified Alpaca prompting to be on a par with ChatML or other conversational formats. This version contains 3x Alpaca modified datasets (for the RP one) and 2 shareGPT datasets. On this v2, LimaRP got fixed further, Axolotl received a monkey patch to actually reformat shareGPT to our modified Alpaca prompting.
- Noromaid-v0.1-mixtral-8x7b-v3
Noromaid-v0.1-mixtral-8x7b-v3 is a fine-tune of Mixtral 8x7b, trained on various RP datasets including LimaRP and Aesir, ToxicDPO without warning for decensoring and norobots, rewritten to use a modified Alpaca prompting to be on a par with ChatML or other conversational formats. This version contains 5x Alpaca modified datasets. On this v3, all the datasets got trained on the "completion" method of Axolotl, with all the datasets being completely rewritten to be in Alpaca modified format. More than 2600 Wikipedia references got cleaned up of Norobots dataset.
- Noromaid-v0.1-mixtral-8x7b-Instruct-v3
Noromaid-v0.1-mixtral-8x7b-Instruct-v3 is made with a LoRA done on base Mixtral 8x7b. It's the same data than Noromaid-v0.1-mixtral-8x7b-v3, but applied on the Instruct model. Fine-tuning on base and applying on Instruct seem to give better result for our usage: RP/ERP. Stay tuned for more information!
Noromaid-v0.1-mixtral-8x7b series info: v1 and v2 were each trained for 2 epochs, totaling to each 8 hours on Axolotl. v3, on the other hand, underwent 3 epochs and was trained for 12 hours, bringing the cumulative training time to 28 hours on a single A100 80GB GPU.
Miqu - [HF Collection]
Quick train to see if miqu finetuned results in good models.
- MiquMaid-v2-70B-alpha-GGUF (GGUF only)
MiquMaid v2-alpha is trained on 1 epoch for 18h running on 2xA100 80GB. Trained on miqu. Deprecated!
- MiquMaid-v2-70B / [DPO]
MiquMaid-v1-70B was our first 70B model, based on the leaked Mistral Medium model. V1 used Aesir datasets where V2 make the return of Norobots and some uncensoring data in it to make it even more unethical in RP.
V1 was already compliant to a lot of things, even on some prompt Mistral Medium would refuse 100%, because it is HIGHLY aligned. This V2 let you prompt even more unethical and unhinged RP. Not using any RP format show a rate of refusal really lower than Mistral Medium too!
On top of that, a DPO train, using the same data that was used in the OG Finetune for better performance made it even better, write better, and be even more uncensored. The model lose some points in benchmark, but the tradeback for really good RP and less repetition was worth it.
- MiquMaid-v2-70B / [DPO]
MiquMaid-v2-2x70B is really heavy, it's a 125B model made of MiquMaid-v2-70B and Mistral Medium base. Each MoE model have 2 expert active in them, so the idea between this was to have, on every token, 2x70B expert working together for more randomness and better precision. Since MiquMaid-v2 is here only for RP, it lose some IQ point, that's where Mistral Medium hit and make the prose a lot more better, and let the model be more logical. It's better than a frakenmerge of 2 Mistral medium because it's not 1:1 2x the same model.
The gem tho, is the DPO version. When the qLora for uncensoring was made on MiquMaid-v2 for uncensoring it, we got the idea to apply it to Mistral Medium base too. At first, the result wasn't really good and that was expected, since the qLora wasn't trained on Mistral Medium, but on MiquMaid-v2. BUT! When merged together (in a MoE for this example) the full potential of a double DPO shine.
We got astonish result on the worst quant ever : Q2_K. Even at Q2_K, MiquMaid-v2-2x70B-DPO showed godly performance in RP, following card, logic and smut. Only downside was the repetition of the formatting, but it was really usable. Unquanted is a gem, but we doubt anyone have the compute power to do that...
(based on 0.4)
- Noromaid-v0.4-Mixtral-Instruct-8x7b-Zloss
It's Noromaid-v0.1-mixtral-8x7b-Instruct-v3 but retrained with ChatML, Zloss(Thanks charles), and some added datasets.
FlatOrcamaid-13b-v0.2 is a merge between FlatOrca and Noromaid-13b-v0.2, see more info on the repo
Important info to a 7b model: Currently not possible due to Orca-2 being a LLAMA 2 7b model and Noromaid-7b being a mistral model!
Important info to this model: This model is in NO way affiliated with ddh0/OrcaMaid-13b or and *maid models by ddh0, it was inspired by him tho.
Credits:
- Undi / Wrote parts of the blog post.
Updates:
20.12.2023 - 11:55 [GMT+2]
Okay so, because some people asked about if the datasets will be released..
Yes they probably will, but not from me neither Undi. The maid part aka. Aesir will only be released after the official Aesir model from MinervaAI is released.
21.12.2023 - 19:56 [GMT+2]
Added FlatOrcamaid-13b-v0.2 [ Released on: 20.12.2023 ]
Added Noromaid-7b-v0.2 [ Released on: 21.12.2023 ]
23.12.2023 - 15:03 [GMT+2]
Added Noromaid-v0.1-mixtral-8x7b [ Released on: 22.12.2023 ]
24.12.2023 - 13:14 [GMT+2]
Added FlatOrcamaid-13b-v0.2 disclaimer
Added Noromaid-v0.1-mixtral-8x7b-v2 [ Released on: 23.12.2023 ]
Added Noromaid-v0.1-mixtral-8x7b-v3 [ Released on: 24.12.2023 ]
26.12.2023 - 00:18 [GMT+2]
Added Noromaid-v0.1-mixtral-8x7b-Instruct-v3 [ Released on: 25.12.2023 ]
09.01.2024 - 20:13 [GMT+2]
Added NeverSleep Discord link
Added Noromaid 0.3 [ Released on: 05.01.2024 ]
12.01.2024 - 19:13 [GMT+2]
Added Noromaid 0.4 [ Released on: 11.01.2024 ]
Added Noromaid 0.4 DPO [ Released on: 11.01.2024 ]
Added credits
Added Noromaid-v0.4-Mixtral-Instruct-8x7b-Zloss [ Released on: 09.01.2024 ]
12.01.2024 - 23:45 [GMT+2]
Did some re-formatting.
Fixed some skill issue mistakes on my side.
05.02.2024 - 20:14 [GMT+2]
Added MiquMaid-v1-70B [ Released on: 31.01.2024 ]
Added MiquMaid-v2-70B-alpha-GGUF [ Released on: 04.02.2024 ]
07.02.2024 - 20:20 [GMT+2]
Added MiquMaid HF Collection
Added MiquMaid-v2-70B [ Released on: 07.02.2024 ]
Added MiquMaid-v2-70B-DPO [ Released on: 07.02.2024 ]
Added MiquMaid-v2-2x70B [ Released on: 07.02.2024 ]
Added MiquMaid-v2-2x70B-DPO [ Released on: 07.02.2024 ]
Collab with Undi / Echidna-13b v0.1, v0.2, v0.3 + NeverSleep/Nethena-13B
27.10.2023 - 19:57 - 23:53 [GMT+2]Echidna-13b-v0.1
Echidna-13b-v0.1 came out 21.10.2023, i wanted to merge it alone first, but had server issues. Thats when i asked Undi to help merge it, we reviewed my recipe and decided to upload it on his NeverSleep org and let it be a collab.
Echidna-13b-v0.2
Echidna-13b-v0.2 came out 26.10.2023 (took 4~ hours of merging), it was kind of a "test" version of Echidna-13b-v0.3. Echidna-13b-v0.3 would follow soon after.
Echidna-13b-v0.3
It took us some time to refine Echidna-13b-v0.2's recipe, we had multiple test models(private) which game way better results than Echidna-13b-v0.2.
The newest version, Echidna-13b-v0.2 came out 27.10.2023 (took 5.30~ hours of merging).
Repetition seems to be almost non-existant, which is really nice. Undi and me noticed that while trying to let the AI talk to itself(with impersonate).
Coherency is outstanding, as is creativity.
This model seems to not have 5 IQ like some other models, for example: a femboy has a dick, not a vagina... It's also really nice at describing things, like scenes. This is probably due to having so much medical data in here.
Last but not least.. not a single "...shiver down X spine..." seen while testing!
Updates:
31.10.2023 - 01:50 [GMT+2]
Nethena-13B
Tbh i cant really keep up with how much models we are posting on here kek..
Well anyways... we released a model called Nethena yesterday which is even better than echidna, this model combines every best model of Undi and me:
Athena v3 + Nete + Echidna v0.3
We also made a Nethena-20B but no one really cares about that.
Please enjoy!
MinervaAI / Aesir
16.09.2023 - 23:31 [GMT+2]Sooo.. hey.. me, Gryphepadar, Doctorshotgun, Lordgoonery, M1xx3l, Nruaif are in sort of(?) a little team making LLM models no.
Currently we are working on a model + dataset called Aesir, we are almost 1 dataset manual cleaning down.
Aesir is meant as a horny-ass RP model(ERP).
You can get more info on our HuggingFace.
Wanna check out my resources page?
14.08.2023 - 23:17 [GMT+2]So i added a little section called "Resources", if you want you can check it out if you wanna look for new stuff programming/ML/LLM/SD or if you are starting out with that stuff.
You can check it out here or you can just click the "resources" button in the Navbar.
One thing i still need to do tho, is making this website better for mobile, as it looks shit currently(i think so at least).
Updates:
I also added a projects page, where i will post future projects.
Website is finally "done"
13.08.2023 - 22:29 [GMT+2]Hey, i finally made the website here have content, and if you are asking "ikari, why tf are you posting so many blog posts, aren't they suposed to be kind of 'special'?"
To that i answer "Shut up, i need to fill this page so it doesn't look empty!"
Anyway.. it now looks awesome! And i removed all the lorem impsum shit, + I added, if you open a post in "full screen" it temporarily removes the sidebar for better visibility of the post.
Updates:
I added a nice looking loading screen, to hide the js magic, and i fixed a couple of things, + quality of life stuff.
One cool thing i added too, was this vote thingy, if you like a post you can vote it up! (you can for some reason press it multiple times, so if you REALLY like a post you could vote it to like 200 or smth)
So i added a couple of cool things
12.08.2023 - 23:05 [GMT+2]I added a new url arg called "bo", when set to true combined with set blogid arg, it opens it in "fullscreen" mode.
I also added a little button(Open post), which does the same. When you click share it automatically sets the blogid arg, + it sets "bo" arg to true.
Hey, this is a little test for the new blog page
12.08.2023 - 04:15 [GMT+2]I will document my jurney as a dev here.
I dont think anyone is even reading this but yea, i dont really care xD
If you wanna for some reason share this or other posts, i added that thing below
Here you can find various resources i am using/i used/i suggest using.
Overview:
ML
LLM
- HuggingFace - NeverSleep ~ Org by me and Undi.
- Ask max ~ If you don't know how something works, ask max!
- Character Tools ~ A collection of tools for creating and managing characters for AI Roleplay.
- Snombler ~ Model ratings.
- BestERP ~ Nice site for model ratings.
- Ayumi ERP rating ~ An Erotic Roleplay model rating by weicon.
- Chub ~ A site full of chars you can download.
- koboldcpp ~ My new main backend for LLM stuff, only for llama.cpp.
- Aphrodite ~ Aphrodite is the official backend engine for PygmalionAI.
- Tabby ~ Backend for exllamav2.
- OobaBooga-textgen webui ~ (old)My main backend for LLM stuff.
- Kalo's sampler post ~ Your sampler settings are probably hurting your model, why?
- HuggingFace - TheBloke ~ here you can find various GGUF quantizations.
- Sillytavern ~ A really good RP/ERP ui. Based on TavernAI, but sillytavern is much better.
- Vali's ChatterUI ~ Basically SillyTavern but hosted on your phone.
- Chatbots Webring ~ Webring for charcard websites.
- HuggingFace - AzureBlack ~ The TheBloke of exl2 quants.
- OSGA ~ OSGA(Open-Source Generative Agents) is a community-driven initiative that builds upon the foundation of 'Generative Agents', for local LLM models. It is based on this Paper(Video).
- Llama.cpp ~ Inference of LLaMA model in pure C/C++. It can run on cpu+gpu, good option if other model loaders require too much vram.
- HuggingFace - Undi / Undi's Kofi ~ A shit-ton of high quality merged models.
- Thoughts guide (kinda outdated) ~ A guide by me on how you can implement characters thoughts into sillytavern.
- annotated-transformer ~ Idk what to write here, but someone said i should link it here.
- Gryphe/MythoMax-L2-13b ~ Was once the best model available for RP/ERP, some still consider it best.
- LambdaLabs ~ Really cost efficient gpu cluster (for example: 1x H100 for $1.99/hr).
Various discords
- NeverSleep ~ NeverSleep server, by me and undi.
- SillyTavern ~ SillyTavern server.
- TheBlock ~ TheBloke server.
- KoboldAI ~ KoboldAI server.
News
- HuggingFace Daily Papers ~ Daily papers.
- 4chan Lmg ~ Sometimes maybe good sometimes maybe sheet.
- LocalLLaMA ~ Subreddit to discuss about Llama, the large language model created by Meta AI. Often really hangs behind news.
Learn
- Neural Networks: Zero to Hero ~ Learning the basics of how AI works.
- Attention is all you need ~ A complete explanation of all the layers of a Transformer Model: Multi-Head Self-Attention, Positional Encoding, including all the matrix multiplications and a complete description of the training and inference process.
Model creation/Dataset Creation
- MergeKit ~ Suite of merging scripts. With those scripts you can merge LLM models with a variety of different merging methods.
- MergeMonster ~ An unsupervised merging algorithm for Transformers-based language models, using a list of phrases (both good and bad) and a fully automated strategy that strives to decrease (or increase) the probability of these phrases occurring in the final merge.
- Quantization script ~ Bunch of Colab/Laggle notebooks for quanting models, applying loras and bunch of other stuff.
- Huggingface Generation config documentation ~ If you use smth like oobabooga, and you dont know what each gen setting does, well... here is the place.
--------------
- Augmentoolkit ~ Generate multi-turn training data, about any subject, using Open Source LLMs.
- airoboros ~ using large language models to fine-tune large language models.
Datasets
- Dampf's list ~ A collection of datasets.
- ERP/RP and erotica rawdata collection ~ A collection of datasets.
- PygmalionAI/PIPPA ~ Official Pygmalion dataset.
- Xilabs/pippa-alpaca ~ Pygmalion PIPPA dataset with alpaca format instead of metharme.
Stable Diffusion
- AUTOMATIC1111 ~ A really good ui.
- vladmandic ~ A fork of AUTOMATIC1111 which does things differently. Try which fits best for you.
- ComfyUI ~ Yet another powerful SD UI, here you have way more control than in vlad, or auto1111. ComfyUI will let you design and execute advanced stable diffusion pipelines using a graph/nodes/flowchart based interface.
- kohya_ss ~ A tool for training sd Loras/Locon/whatever.
- CivitAi ~ A place where you can get various Öoras/models/whatever.
- YameroYandere Lora ~ A really good Yandere lora, made by me with the around 500 images.
Web development
Neocities
Neocities type of sites.
- sadgrl.online ~ Bunch of stuff neocities related.
- y2k ~ Like the last one but not sorted.
- gifcities ~ Archive of neocities-like gifs you can use on your website.
- ezgif ~ A really good online tool for modifying gifs.