Resources

Presentation materials and surveys used during the study sessions are available to the public. Note that the contents are mainly written in Japanese.

Additionally, we are updating the Overview of Japanese LLMs.

• 2024-10-29
  • LLM-jp Status Report (Kurohashi)
  • BritLLM:Organising, producing, and publishing the first British Large Language Model (Pontus Stenetorp/NII)
  • Pre-training and Post-training of PLaMo100B (Hiroaki Mikami,Kosuke Nakago/Preferred Elements)
  • Multi-modal Working Group (Okazaki,Sasagawa,Maeda,Sugiura)
  • Model Building Working Group (Suzuki)
  • Corpus Construction Working Group (Kawahara)
  • Evaluation and Tuning Working Group (Miyao)
  • Safety Working Group (Sekine)
  • Real-world Interaction Working Group (Ogata)
• 2024-08-27
  • On Web Article Crawling and Copyright Infringement (Kakinuma)
  • Beyond English-Centric LLMs: What Language Do Multilingual Language Models Think in? (Qianying Liu/NII)
  • Real-world Interaction Working Group (Ogata)
  • Corpus Construction Working Group (Kawahara)
  • Evaluation and Tuning Working Group (Miyao)
  • Safety Working Group (Sekine)
  • Multi-modal Working Group (Okazaki)
  • Model Building Working Group (Suzuki)
• 2024-07-30
  • LLM-jp Status Report (Kurohashi)
  • What is Open Source AI? Explaining the Draft Version of “The Open Source AI Definition” (Sado)
  • Recent Advances in Addressing Hallucinations (Tsuta)
  • Corpus Construction Working Group (Kawahara)
  • Safety Working Group (Sekine)
  • Evaluation and Tuning Working Group (Miyao)
  • Multi-modal Working Group (Okazaki)
  • Report on the Setup of Sakura Cluster (Kuga)
  • Model Building Working Group (Suzuki)
• 2024-06-25
  • LLM-jp Status Report (Kurohashi)
  • Latest Advancements in Document Image Understanding with Large Language Models (Tanaka)
  • Development of Nejumi Leaderboard3 (Kamata)
  • Sarashina: Introduction to Japanese LLMs developed by SB Intuitions (Takase)
  • Mechanistic Interpretability: Introduction to Scaling Monosemanticity (Anthropic, 2024) (Takagi)
  • Corpus Construction Working Group (Kawahara)
  • Model Building Working Group (Suzuki)
  • Evaluation and Tuning Working Group (Miyao)
  • Safety Working Group (Sekine)
  • Multi-modal Working Group (Okazaki)
• 2024-05-26
  • LLM-jp Status Report (Kurohashi)
  • Development of a Japanese LLM with 100B Parameters (Omi)
  • Overview of R&D for Generative AI at Databricks (Yayoi)
  • Multilingual LLM: Data Construction, Fine-tuning, and LLM-based Evaluation (Peter)
  • Corpus Construction Working Group (Kawahara)
  • Model Building Working Group (Suzuki)
  • Evaluation and Tuning Working Group (Miyao)
  • Safety Working Group (Sekine)
  • Multi-modal Working Group (Okazaki)
• 2024-03-26
  • LLM-jp Status Report (Kurohashi)
  • Geniac Program Initiatives at ABEJA (Hattori)
  • Estimated timing of transition from Prompt Tuning to Fine Tuning (Kubo)
  • An experiment to reduce halucinations produced by ichikara-instruction, Less NE experiments (Sekine)
  • Evaluation and Tuning Working Group (Miyao)
  • Corpus Construction Working Group (Kawahara)
  • Safety Working Group (Sekine)
  • Model Building Working Group (Suzuki)
• 2024-01-22
  • Self-improvement loops from observational data, Grounding LLMs with causal inference methods (Sannai)
  • Comparative analysis of LLM human evaluation and GPT evaluation using ichikara-instruction (Sekine)
  • Kotoba Technologies voice-based model and Mamba architecture (Kojima and Kasai)
  • Large language model, Swallow (TokyoTech LLM members)
  • Safety Working Group (Sekine)
  • Corpus Construction Working Group (Kawahara)
  • Evaluation and Tuning Working Group (Miyao)
  • Model Building Working Group (Suzuki)
• 2023-11-29
  • LLM-jp Status Report (Kurohashi)
  • LLM Security Survey and Japanese Data (Suzuki)
  • Development of 13 Billion Parameter Japanese Pre-Learning Model for Business Domains and Latest Information (Omi)
  • Potential and Progress of Large-scale Language Models in Medicine (Kodera)
  • Corpus Construction Working Group (Kawahara)
  • Model Building Working Group (Suzuki)
  • Evaluation and Tuning Working Group (Miyao)
  • Safety Working Group (Sekine)
• 2023-10-18
  • Training and applying Vision & Language model with Heron (Inoue)
  • Technical overview of Japanese LLM “ELYZA-japanese-Llama-2-7b” based on “Llama 2” from Meta (Nakamura)
  • Technical overview of pre-training for PLaMo-13B (Mikami)
  • LLM models for robotics (Kawaharazuka)
• 2023-09-04
  • A survey for instruction tuning for Model imitation explained (Mizuki)
  • PEFT: LazyLoRA (Wu)
  • Overview of R&D for LLM at Stability AI Japan (Lee)
  • Generative AI, construction and copyright (Kakinuma)
  • Construction of a large-scale bilingual English/Japanese language model “Weblab-10B” (Kojima)
• 2023-07-20
  • ACL2023 participation announcement (Kodama) (Yamada) (Ueda) (Deguchi)
  • LLM Peripheral technology (June/July 2023) (Tsuruoka)
• 2023-06-19
  • Overview of the model project for CyberAgent (Ishigami)
  • Tips for training of T5 in Japanese (Nishitoba)
  • Participation report for the ABCI Grand challenge (Sakaguchi)
  • Research and development of large-scale language models at NICT (Torizawa)​
  • Future Cooperation for NII：Domain adaptation for Biomedical (Aizawa)
• 2023-05-15
  • PaLM2 Technical Report - Thesis Introduction (Okazaki)
  • Toward the co-evolution of Japanese basic models and language understanding benchmarks (Kawahara)
  • Survey about LLM (Kawahara, Sugawara, and Kurita)