AIEdx is building open small language models from scratch.

BubblSpace is our open research effort at AIEdx. We pretrain sparse mixture-of-experts models from first principles and publish every artifact—weights, intermediate checkpoints, data pipelines, live training runs, and research notes—back to the community.

BubblSpace MoE-0.9B-A0.2B

915M total · 211M active · 10B tokens

Our current open run is BubblSpace MoE-0.9B-A0.2B, a Qwen3-Next-inspired dropless top-8 SwiGLU mixture-of-experts base model (64 experts, 16 layers, GQA 16/4, QK-norm, RoPE, ZeroCenteredRMSNorm, tied embeddings) trained with Muon + AdamW in bf16 on 10B tokens of FineWeb-Edu on a single H100.

Mid-training checkpoints (lm-evaluation-harness, base model · 19,073 planned steps)

BenchmarkMetricstep_7000step_8000step_9000
PIQAacc_norm64.7%64.5%65.9%
ARC-Easyacc_norm51.1%50.7%50.8%
HellaSwagacc_norm37.0%38.0%38.5%
LAMBADAperplexity58.150.349.3

Open means we share everything

  • All model weights and intermediate checkpoints on HuggingFace
  • Full model architecture, training loop, tokenizer, and FineWeb-Edu data-prep code on GitHub
  • Public live training curves and diagnostics on Weights & Biases
  • Provisional accounts of experiments, ablations, and evals in Research Notes

Recent Research Notes

All notes →

Team

Amardeep Singh Sidhu

Founder & AI Researcher

First-principles builder and researcher behind AIEdx and the BubblSpace MoE pretraining runs.

Private
PrivateFablePrime

Co-Founder & Strategy

Co-founder supporting strategy, operations, and community growth at AIEdx.

BubblSpace is developed at AIEdx. To collaborate, discuss an experiment, or follow up on a checkpoint, reach out via Contact Us or email contact@aiedx.com.