By Selim Özten - Lead Artificial Intelligence Engineer
At inpocket.ai, we're constantly pushing the boundaries of AI-generated art for educational purposes. Today, I'm excited to share the details of our latest project: the Sida Hand Drawn LoRA model, specifically designed for our upcoming educational game, "Learn Isle."
The Vision
Our goal was to create a model that could generate hand-drawn style illustrations consistently, capturing the unique artistic flair of our talented artist, Sida. These illustrations will form the visual backbone of "Learn Isle," engaging children with friendly, educational content.
The Model
We're proud to introduce Sida Hand Drawn 0.1 (sida-hd-01) a LoRA (Low-Rank Adaptation) model fine-tuned on the powerful FLUX.1-dev base model. This combination allows us to generate high-quality, hand-drawn style images that perfectly suit our educational needs.
The Training Process
Dataset
We curated a small but highly focused dataset of 11 hand-drawn images by Sida. Quality over quantity was our mantra, ensuring each image captured the essence of the style we wanted to reproduce.
Image Captioning
To provide a rich context for our model, we used the state-of-the-art Florence2 model to caption our images. This step was crucial in helping the model understand the nuances of each illustration.
Training Configuration
We used the ostris-ai-toolkit for training, which provided us with a flexible and powerful framework. Here are some key details from our training configuration: https://huggingface.co/inpocketai/sida-hd-01/blob/main/training_config.yaml
- LoRA Rank: 32
- Training Steps: 3000
- Learning Rate: 1e-4
- Batch Size: 1
- Resolution: [512, 768, 1024]
- Scheduler: FlowMatch
- Optimizer: AdamW8bit
- Precision: bfloat16
Hardware and Time
We conducted the training on a single NVIDIA RTX 4090 with 24GB of VRAM. The entire process took approximately 1.5 hours, demonstrating the efficiency of our setup and the LoRA fine-tuning approach.
Results and Usage
The resulting model excels at generating hand-drawn style illustrations when prompted with "hand-drawn illustration of a" at the beginning of the input. It's particularly adept at creating educational, child-friendly content that aligns perfectly with our vision for "Learn Isle."
You can find the model files and try it out yourself on our Hugging Face repository. We've also included sample images in the repository to showcase the model's capabilities.
Looking Ahead
This project represents a significant step forward in our ability to create consistent, high-quality illustrations for educational content. As we continue to develop "Learn Isle," the Sida Hand Drawn model will play a crucial role in bringing our educational concepts to life.
We're excited about the possibilities this opens up for creating engaging, visually appealing educational materials. Stay tuned for more updates as we continue to innovate at the intersection of AI, art, and education.
For more information about Learn Isle, inpocket.ai, and our other projects, visit our website or contact our team.
What are your thoughts on AI-generated art in educational contexts? We'd love to hear your perspective in the comments below!
留言