下载资源人工智能资源详情

minimind-master.zip

大小:3.59MB

价格:36积分

下载量:0

评分:

5.0

上传者:yzj5464

更新日期:2025-09-22

minimind最小开源大模型，可以自己训练自己的大模型

资源文件列表(大概)

文件名

大小

minimind-master/

minimind-master/0-eval_pretrain.py

5.72KB

minimind-master/1-pretrain.py

6.17KB

minimind-master/2-eval.py

6.18KB

minimind-master/3-full_sft.py

6.95KB

minimind-master/4-lora_sft.py

5.17KB

minimind-master/5-dpo_train.py

2.21KB

minimind-master/CODE_OF_CONDUCT.md

5.08KB

minimind-master/LICENSE

11.09KB

minimind-master/README.md

48.78KB

minimind-master/README_en.md

50.21KB

minimind-master/ceval/

minimind-master/ceval/ceval-exam/

minimind-master/ceval/ceval-exam/dev/

minimind-master/ceval/ceval-exam/dev/accountant_dev.csv

3.26KB

minimind-master/ceval/ceval-exam/dev/advanced_mathematics_dev.csv

6.79KB

minimind-master/ceval/ceval-exam/dev/art_studies_dev.csv

1.33KB

minimind-master/ceval/ceval-exam/dev/basic_medicine_dev.csv

1.71KB

minimind-master/ceval/ceval-exam/dev/business_administration_dev.csv

3.02KB

minimind-master/ceval/ceval-exam/dev/chinese_language_and_literature_dev.csv

1.78KB

minimind-master/ceval/ceval-exam/dev/civil_servant_dev.csv

4.4KB

minimind-master/ceval/ceval-exam/dev/clinical_medicine_dev.csv

1.83KB

minimind-master/ceval/ceval-exam/dev/college_chemistry_dev.csv

3.46KB

minimind-master/ceval/ceval-exam/dev/college_economics_dev.csv

3.52KB

minimind-master/ceval/ceval-exam/dev/college_physics_dev.csv

3.67KB

minimind-master/ceval/ceval-exam/dev/college_programming_dev.csv

2.75KB

minimind-master/ceval/ceval-exam/dev/computer_architecture_dev.csv

2.66KB

minimind-master/ceval/ceval-exam/dev/computer_network_dev.csv

2.23KB

minimind-master/ceval/ceval-exam/dev/discrete_mathematics_dev.csv

1.89KB

minimind-master/ceval/ceval-exam/dev/education_science_dev.csv

2.95KB

minimind-master/ceval/ceval-exam/dev/electrical_engineer_dev.csv

2.05KB

minimind-master/ceval/ceval-exam/dev/environmental_impact_assessment_engineer_dev.csv

2.36KB

minimind-master/ceval/ceval-exam/dev/fire_engineer_dev.csv

2.08KB

minimind-master/ceval/ceval-exam/dev/high_school_biology_dev.csv

2.04KB

minimind-master/ceval/ceval-exam/dev/high_school_chemistry_dev.csv

2.45KB

minimind-master/ceval/ceval-exam/dev/high_school_chinese_dev.csv

5.1KB

minimind-master/ceval/ceval-exam/dev/high_school_geography_dev.csv

1.97KB

minimind-master/ceval/ceval-exam/dev/high_school_history_dev.csv

2.3KB

minimind-master/ceval/ceval-exam/dev/high_school_mathematics_dev.csv

3.41KB

minimind-master/ceval/ceval-exam/dev/high_school_physics_dev.csv

2.14KB

minimind-master/ceval/ceval-exam/dev/high_school_politics_dev.csv

4.56KB

minimind-master/ceval/ceval-exam/dev/ideological_and_moral_cultivation_dev.csv

1.19KB

minimind-master/ceval/ceval-exam/dev/law_dev.csv

3.98KB

minimind-master/ceval/ceval-exam/dev/legal_professional_dev.csv

6.75KB

minimind-master/ceval/ceval-exam/dev/logic_dev.csv

5.45KB

minimind-master/ceval/ceval-exam/dev/mao_zedong_thought_dev.csv

3.2KB

minimind-master/ceval/ceval-exam/dev/marxism_dev.csv

2.02KB

minimind-master/ceval/ceval-exam/dev/metrology_engineer_dev.csv

2.35KB

minimind-master/ceval/ceval-exam/dev/middle_school_biology_dev.csv

4.16KB

minimind-master/ceval/ceval-exam/dev/middle_school_chemistry_dev.csv

3.71KB

minimind-master/ceval/ceval-exam/dev/middle_school_geography_dev.csv

2.03KB

minimind-master/ceval/ceval-exam/dev/middle_school_history_dev.csv

1.89KB

minimind-master/ceval/ceval-exam/dev/middle_school_mathematics_dev.csv

3.05KB

minimind-master/ceval/ceval-exam/dev/middle_school_physics_dev.csv

3.38KB

minimind-master/ceval/ceval-exam/dev/middle_school_politics_dev.csv

3.54KB

minimind-master/ceval/ceval-exam/dev/modern_chinese_history_dev.csv

2.84KB

minimind-master/ceval/ceval-exam/dev/operating_system_dev.csv

2.42KB

minimind-master/ceval/ceval-exam/dev/physician_dev.csv

1.91KB

minimind-master/ceval/ceval-exam/dev/plant_protection_dev.csv

3.57KB

minimind-master/ceval/ceval-exam/dev/probability_and_statistics_dev.csv

6.55KB

minimind-master/ceval/ceval-exam/dev/professional_tour_guide_dev.csv

1.65KB

minimind-master/ceval/ceval-exam/dev/sports_science_dev.csv

4.02KB

minimind-master/ceval/ceval-exam/dev/tax_accountant_dev.csv

4.11KB

minimind-master/ceval/ceval-exam/dev/teacher_qualification_dev.csv

3.07KB

minimind-master/ceval/ceval-exam/dev/urban_and_rural_planner_dev.csv

3.02KB

minimind-master/ceval/ceval-exam/dev/veterinary_medicine_dev.csv

2.24KB

minimind-master/ceval/ceval-exam/test/

minimind-master/ceval/ceval-exam/test/accountant_test.csv

162.73KB

minimind-master/ceval/ceval-exam/test/advanced_mathematics_test.csv

45.05KB

minimind-master/ceval/ceval-exam/test/art_studies_test.csv

33.48KB

minimind-master/ceval/ceval-exam/test/basic_medicine_test.csv

24.13KB

minimind-master/ceval/ceval-exam/test/business_administration_test.csv

69.71KB

minimind-master/ceval/ceval-exam/test/chinese_language_and_literature_test.csv

26.79KB

minimind-master/ceval/ceval-exam/test/civil_servant_test.csv

167.53KB

minimind-master/ceval/ceval-exam/test/clinical_medicine_test.csv

36.59KB

minimind-master/ceval/ceval-exam/test/college_chemistry_test.csv

39.81KB

minimind-master/ceval/ceval-exam/test/college_economics_test.csv

105.69KB

minimind-master/ceval/ceval-exam/test/college_physics_test.csv

50.42KB

minimind-master/ceval/ceval-exam/test/college_programming_test.csv

74.42KB

minimind-master/ceval/ceval-exam/test/computer_architecture_test.csv

35.24KB

minimind-master/ceval/ceval-exam/test/computer_network_test.csv

30.67KB

minimind-master/ceval/ceval-exam/test/discrete_mathematics_test.csv

31.85KB

minimind-master/ceval/ceval-exam/test/education_science_test.csv

48.29KB

minimind-master/ceval/ceval-exam/test/electrical_engineer_test.csv

64.3KB

minimind-master/ceval/ceval-exam/test/environmental_impact_assessment_engineer_test.csv

76.3KB

minimind-master/ceval/ceval-exam/test/fire_engineer_test.csv

75.23KB

minimind-master/ceval/ceval-exam/test/high_school_biology_test.csv

49.98KB

minimind-master/ceval/ceval-exam/test/high_school_chemistry_test.csv

41.91KB

minimind-master/ceval/ceval-exam/test/high_school_chinese_test.csv

103.67KB

minimind-master/ceval/ceval-exam/test/high_school_geography_test.csv

36.2KB

minimind-master/ceval/ceval-exam/test/high_school_history_test.csv

50.77KB

minimind-master/ceval/ceval-exam/test/high_school_mathematics_test.csv

36.69KB

minimind-master/ceval/ceval-exam/test/high_school_physics_test.csv

56.34KB

minimind-master/ceval/ceval-exam/test/high_school_politics_test.csv

77.54KB

minimind-master/ceval/ceval-exam/test/ideological_and_moral_cultivation_test.csv

30.55KB

minimind-master/ceval/ceval-exam/test/law_test.csv

72.86KB

minimind-master/ceval/ceval-exam/test/legal_professional_test.csv

114.43KB

minimind-master/ceval/ceval-exam/test/logic_test.csv

136.26KB

minimind-master/ceval/ceval-exam/test/mao_zedong_thought_test.csv

50.38KB

minimind-master/ceval/ceval-exam/test/marxism_test.csv

33.65KB

minimind-master/ceval/ceval-exam/test/metrology_engineer_test.csv

41.37KB

minimind-master/ceval/ceval-exam/test/middle_school_biology_test.csv

41.76KB

minimind-master/ceval/ceval-exam/test/middle_school_chemistry_test.csv

42.22KB

minimind-master/ceval/ceval-exam/test/middle_school_geography_test.csv

20.28KB

minimind-master/ceval/ceval-exam/test/middle_school_history_test.csv

41.23KB

minimind-master/ceval/ceval-exam/test/middle_school_mathematics_test.csv

28.41KB

minimind-master/ceval/ceval-exam/test/middle_school_physics_test.csv

43.57KB

minimind-master/ceval/ceval-exam/test/middle_school_politics_test.csv

66.53KB

minimind-master/ceval/ceval-exam/test/modern_chinese_history_test.csv

45.2KB

minimind-master/ceval/ceval-exam/test/operating_system_test.csv

26.32KB

minimind-master/ceval/ceval-exam/test/physician_test.csv

77.67KB

minimind-master/ceval/ceval-exam/test/plant_protection_test.csv

26.57KB

minimind-master/ceval/ceval-exam/test/probability_and_statistics_test.csv

52.17KB

minimind-master/ceval/ceval-exam/test/professional_tour_guide_test.csv

34.2KB

minimind-master/ceval/ceval-exam/test/sports_science_test.csv

27.63KB

minimind-master/ceval/ceval-exam/test/tax_accountant_test.csv

160.36KB

minimind-master/ceval/ceval-exam/test/teacher_qualification_test.csv

95.84KB

minimind-master/ceval/ceval-exam/test/urban_and_rural_planner_test.csv

98.31KB

minimind-master/ceval/ceval-exam/test/veterinary_medicine_test.csv

33.79KB

minimind-master/ceval/ceval-exam/val/

minimind-master/ceval/ceval-exam/val/accountant_val.csv

18.01KB

minimind-master/ceval/ceval-exam/val/advanced_mathematics_val.csv

4.83KB

minimind-master/ceval/ceval-exam/val/art_studies_val.csv

3.75KB

minimind-master/ceval/ceval-exam/val/basic_medicine_val.csv

2.16KB

minimind-master/ceval/ceval-exam/val/business_administration_val.csv

8.28KB

minimind-master/ceval/ceval-exam/val/chinese_language_and_literature_val.csv

2.87KB

minimind-master/ceval/ceval-exam/val/civil_servant_val.csv

19.74KB

minimind-master/ceval/ceval-exam/val/clinical_medicine_val.csv

3.59KB

minimind-master/ceval/ceval-exam/val/college_chemistry_val.csv

3.83KB

minimind-master/ceval/ceval-exam/val/college_economics_val.csv

12.91KB

minimind-master/ceval/ceval-exam/val/college_physics_val.csv

5.6KB

minimind-master/ceval/ceval-exam/val/college_programming_val.csv

8.58KB

minimind-master/ceval/ceval-exam/val/computer_architecture_val.csv

3.6KB

minimind-master/ceval/ceval-exam/val/computer_network_val.csv

3.3KB

minimind-master/ceval/ceval-exam/val/discrete_mathematics_val.csv

3.01KB

minimind-master/ceval/ceval-exam/val/education_science_val.csv

4.75KB

minimind-master/ceval/ceval-exam/val/electrical_engineer_val.csv

7.31KB

minimind-master/ceval/ceval-exam/val/environmental_impact_assessment_engineer_val.csv

8.29KB

minimind-master/ceval/ceval-exam/val/fire_engineer_val.csv

9.08KB

minimind-master/ceval/ceval-exam/val/high_school_biology_val.csv

5.56KB

minimind-master/ceval/ceval-exam/val/high_school_chemistry_val.csv

5.08KB

minimind-master/ceval/ceval-exam/val/high_school_chinese_val.csv

9.83KB

minimind-master/ceval/ceval-exam/val/high_school_geography_val.csv

3.49KB

minimind-master/ceval/ceval-exam/val/high_school_history_val.csv

6.04KB

minimind-master/ceval/ceval-exam/val/high_school_mathematics_val.csv

4.68KB

minimind-master/ceval/ceval-exam/val/high_school_physics_val.csv

6.69KB

minimind-master/ceval/ceval-exam/val/high_school_politics_val.csv

8.31KB

minimind-master/ceval/ceval-exam/val/ideological_and_moral_cultivation_val.csv

2.76KB

minimind-master/ceval/ceval-exam/val/law_val.csv

7.41KB

minimind-master/ceval/ceval-exam/val/legal_professional_val.csv

11.43KB

minimind-master/ceval/ceval-exam/val/logic_val.csv

14.72KB

minimind-master/ceval/ceval-exam/val/mao_zedong_thought_val.csv

4.84KB

minimind-master/ceval/ceval-exam/val/marxism_val.csv

3.74KB

minimind-master/ceval/ceval-exam/val/metrology_engineer_val.csv

5.45KB

minimind-master/ceval/ceval-exam/val/middle_school_biology_val.csv

4.68KB

minimind-master/ceval/ceval-exam/val/middle_school_chemistry_val.csv

5.09KB

minimind-master/ceval/ceval-exam/val/middle_school_geography_val.csv

2.33KB

minimind-master/ceval/ceval-exam/val/middle_school_history_val.csv

5.37KB

minimind-master/ceval/ceval-exam/val/middle_school_mathematics_val.csv

4.4KB

minimind-master/ceval/ceval-exam/val/middle_school_physics_val.csv

4.75KB

minimind-master/ceval/ceval-exam/val/middle_school_politics_val.csv

6.69KB

minimind-master/ceval/ceval-exam/val/modern_chinese_history_val.csv

4.57KB

minimind-master/ceval/ceval-exam/val/operating_system_val.csv

2.81KB

minimind-master/ceval/ceval-exam/val/physician_val.csv

7.42KB

minimind-master/ceval/ceval-exam/val/plant_protection_val.csv

3.07KB

minimind-master/ceval/ceval-exam/val/probability_and_statistics_val.csv

5.31KB

minimind-master/ceval/ceval-exam/val/professional_tour_guide_val.csv

3.77KB

minimind-master/ceval/ceval-exam/val/sports_science_val.csv

3KB

minimind-master/ceval/ceval-exam/val/tax_accountant_val.csv

17.4KB

minimind-master/ceval/ceval-exam/val/teacher_qualification_val.csv

10.97KB

minimind-master/ceval/ceval-exam/val/urban_and_rural_planner_val.csv

11.48KB

minimind-master/ceval/ceval-exam/val/veterinary_medicine_val.csv

3.97KB

minimind-master/ceval/ceval_result/

minimind-master/ceval/ceval_result/accountant_val_result.csv

18.14KB

minimind-master/ceval/ceval_result/advanced_mathematics_val_result.csv

4.94KB

minimind-master/ceval/ceval_result/art_studies_val_result.csv

3.87KB

minimind-master/ceval/ceval_result/basic_medicine_val_result.csv

2.26KB

minimind-master/ceval/ceval_result/business_administration_val_result.csv

8.42KB

minimind-master/ceval/ceval_result/chinese_language_and_literature_val_result.csv

2.99KB

minimind-master/ceval/ceval_result/civil_servant_val_result.csv

19.87KB

minimind-master/ceval/ceval_result/clinical_medicine_val_result.csv

3.71KB

minimind-master/ceval/ceval_result/college_chemistry_val_result.csv

3.95KB

minimind-master/ceval/ceval_result/college_economics_val_result.csv

13.05KB

minimind-master/ceval/ceval_result/college_physics_val_result.csv

5.71KB

minimind-master/ceval/ceval_result/college_programming_val_result.csv

8.71KB

minimind-master/ceval/ceval_result/computer_architecture_val_result.csv

3.71KB

minimind-master/ceval/ceval_result/computer_network_val_result.csv

3.41KB

minimind-master/ceval/ceval_result/discrete_mathematics_val_result.csv

3.12KB

minimind-master/ceval/ceval_result/education_science_val_result.csv

4.88KB

minimind-master/ceval/ceval_result/electrical_engineer_val_result.csv

7.44KB

minimind-master/ceval/ceval_result/environmental_impact_assessment_engineer_val_result.csv

8.44KB

minimind-master/ceval/ceval_result/fire_engineer_val_result.csv

9.2KB

minimind-master/ceval/ceval_result/high_school_biology_val_result.csv

5.67KB

minimind-master/ceval/ceval_result/high_school_chemistry_val_result.csv

5.2KB

minimind-master/ceval/ceval_result/high_school_chinese_val_result.csv

9.94KB

minimind-master/ceval/ceval_result/high_school_geography_val_result.csv

3.6KB

minimind-master/ceval/ceval_result/high_school_history_val_result.csv

6.15KB

minimind-master/ceval/ceval_result/high_school_mathematics_val_result.csv

4.8KB

minimind-master/ceval/ceval_result/high_school_physics_val_result.csv

6.8KB

minimind-master/ceval/ceval_result/high_school_politics_val_result.csv

8.43KB

minimind-master/ceval/ceval_result/ideological_and_moral_cultivation_val_result.csv

2.88KB

minimind-master/ceval/ceval_result/law_val_result.csv

7.51KB

minimind-master/ceval/ceval_result/legal_professional_val_result.csv

11.54KB

minimind-master/ceval/ceval_result/logic_val_result.csv

14.83KB

minimind-master/ceval/ceval_result/mao_zedong_thought_val_result.csv

4.95KB

minimind-master/ceval/ceval_result/marxism_val_result.csv

3.84KB

minimind-master/ceval/ceval_result/metrology_engineer_val_result.csv

5.57KB

minimind-master/ceval/ceval_result/middle_school_biology_val_result.csv

4.8KB

minimind-master/ceval/ceval_result/middle_school_chemistry_val_result.csv

5.21KB

minimind-master/ceval/ceval_result/middle_school_geography_val_result.csv

2.44KB

minimind-master/ceval/ceval_result/middle_school_history_val_result.csv

5.49KB

minimind-master/ceval/ceval_result/middle_school_mathematics_val_result.csv

4.52KB

minimind-master/ceval/ceval_result/middle_school_physics_val_result.csv

4.86KB

minimind-master/ceval/ceval_result/middle_school_politics_val_result.csv

6.8KB

minimind-master/ceval/ceval_result/modern_chinese_history_val_result.csv

4.69KB

minimind-master/ceval/ceval_result/operating_system_val_result.csv

2.92KB

minimind-master/ceval/ceval_result/physician_val_result.csv

7.55KB

minimind-master/ceval/ceval_result/plant_protection_val_result.csv

3.18KB

minimind-master/ceval/ceval_result/probability_and_statistics_val_result.csv

5.43KB

minimind-master/ceval/ceval_result/professional_tour_guide_val_result.csv

3.9KB

minimind-master/ceval/ceval_result/sports_science_val_result.csv

3.11KB

minimind-master/ceval/ceval_result/tax_accountant_val_result.csv

17.54KB

minimind-master/ceval/ceval_result/teacher_qualification_val_result.csv

11.11KB

minimind-master/ceval/ceval_result/test.log

4.34KB

minimind-master/ceval/ceval_result/urban_and_rural_planner_val_result.csv

11.62KB

minimind-master/ceval/ceval_result/veterinary_medicine_val_result.csv

4.09KB

minimind-master/chat_openai_api.py

1.42KB

minimind-master/data_process.py

6.62KB

minimind-master/eval_ceval.py

6.87KB

minimind-master/export_model.py

2.07KB

minimind-master/fast_infenence.py

4.2KB

minimind-master/images/

minimind-master/images/1-wiki.png

136.21KB

minimind-master/images/2-eval.png

109.16KB

minimind-master/images/2-wiki.png

72.93KB

minimind-master/images/3-wiki.png

229.81KB

minimind-master/images/4-wiki.png

104.37KB

minimind-master/images/5-wiki.png

239.4KB

minimind-master/images/LLM-structure-moe.png

153.43KB

minimind-master/images/LLM-structure.png

133.48KB

minimind-master/images/fastgpt.png

72.41KB

minimind-master/images/gpt3_config.png

65.63KB

minimind-master/images/logger.png

81.37KB

minimind-master/images/logo.png

402KB

minimind-master/images/streamlit.png

38.81KB

minimind-master/model/

minimind-master/model/LMConfig.py

2.25KB

minimind-master/model/__pycache__/

minimind-master/model/__pycache__/LMConfig.cpython-310.pyc

1.49KB

minimind-master/model/__pycache__/dataset.cpython-310.pyc

3.58KB

minimind-master/model/__pycache__/model.cpython-310.pyc

15.57KB

minimind-master/model/dataset.py

4.23KB

minimind-master/model/minimind_tokenizer/

minimind-master/model/minimind_tokenizer/merges.txt

56.41KB

minimind-master/model/minimind_tokenizer/tokenizer.json

255.29KB

minimind-master/model/minimind_tokenizer/tokenizer_config.json

1.57KB

minimind-master/model/minimind_tokenizer/vocab.json

93.75KB

minimind-master/model/model.py

16.53KB

minimind-master/my_openai_api.py

15.03KB

minimind-master/requirements.txt

425B

minimind-master/train_tokenizer.py

5.26KB

资源内容介绍

minimind最小开源大模型，可以自己训练自己的大模型

<div align="center">![logo](./images/logo.png)</div><div align="center">![visitors](https://visitor-badge.laobi.icu/badge?page_id=jingyaogong/minimind)[![GitHub Repo stars](https://img.shields.io/github/stars/jingyaogong/minimind?style=social)](https://github.com/jingyaogong/minimind/stargazers)[![GitHub Code License](https://img.shields.io/github/license/jingyaogong/minimind)](LICENSE)[![GitHub last commit](https://img.shields.io/github/last-commit/jingyaogong/minimind)](https://github.com/jingyaogong/minimind/commits/master)[![GitHub pull request](https://img.shields.io/badge/PRs-welcome-blue)](https://github.com/jingyaogong/minimind/pulls)[![Collection](https://img.shields.io/badge/🤗-MiniMind%20%20Collection-blue)](https://huggingface.co/collections/jingyaogong/minimind-66caf8d999f5c7fa64f399e5)</div><div align="center"> <h3>"大道至简"</h3></div><div align="center">中文 | [English](./README_en.md)</div>* 本开源项目旨在完全从0开始，最快仅用3小时！即可训练出仅为26M大小的微型语言模型**MiniMind**。* **MiniMind**极其轻量，体积约是 GPT3 的 $\frac{1}{7000}$，力求做到最普通的个人GPU也可快速推理甚至训练。* **MiniMind**改进自DeepSeek-V2、Llama3结构，项目包含整个数据处理、pretrain、sft、dpo的全部阶段，包含混合专家(MoE)模型。* 这是一个既是开源项目，又是入门LLM教程，同时也是一个初具雏形的开源模型，希望能起到抛砖引玉的作用。---<div align="center">https://github.com/user-attachments/assets/88b98128-636e-43bc-a419-b1b1403c2055[Bilibili视频链接](https://www.bilibili.com/video/BV12dHPeqE72/?share_source=copy_web&vd_source=670c2504f88726f8cf4a21ef6147c0e8)</div># 📌 Introduction大语言模型（LLM）领域，如 GPT、LLaMA、GLM 等，虽然它们效果惊艳，但动辄10 Bilion庞大的模型参数个人设备显存远不够训练，甚至推理困难。几乎所有人都不会只满足于用Lora等方案fine-tuing大模型学会一些新的指令，这约等于在教牛顿玩21世纪的智能手机，然而，这远远脱离了学习物理本身的奥妙。此外，卖课付费订阅的营销号漏洞百出的一知半解讲解AI的教程遍地，让理解LLM的优质内容雪上加霜，严重阻碍了学习者。因此，本项目的目标是把上手LLM的门槛无限降低，直接从0开始训练一个极其轻量的语言模型。> [!TIP]> （截至2024-9-17）minimind训练了3个型号模型，最小仅需26M（0.02B），即可具备流畅的对话能力！| 模型 (大小) | tokenizer长度 | 推理占用 | release | 主观评分（/100） | |-------------------------|-------------|--------|------------|------------|| minimind-v1-small (26M) | 6400 | 0.5 GB | 2024.08.28 | 50' || minimind-v1-moe (4×26M) | 6400 | 1.0 GB | 2024.09.17 | 55' || minimind-v1 (108M) | 6400 | 1.0 GB | 2024.09.01 | 60' |> 该分析在一个带有Torch 2.1.2、CUDA 12.2和Flash Attention 2的RTX 3090 GPU上运行。项目包含：- 公开MiniMind模型代码（包含Dense和MoE模型）、Pretrain、SFT指令微调、LoRA微调、DPO偏好优化的全过程代码、数据集和来源。- 兼容`transformers`、`accelerate`、`trl`、`peft`等流行框架。- 训练支持单机单卡、单机多卡(DDP、DeepSpeed)训练。训练过程中支持在任意位置停止，及在任意位置继续训练。- 在Ceval数据集上进行模型测试的代码。- 实现Openai-Api基本的chat接口，便于集成到第三方ChatUI使用（FastGPT、Open-WebUI等）。希望此开源项目可以帮助LLM初学者快速入门！### 👉**最近更新**<details close> <summary> <b>2024-09-17 (new🎉)</b> </summary>- 更新minimind-v1-moe模型- 为了防止歧义，不再使用mistral_tokenizer分词，全部采用自定义的minimind_tokenizer作为分词器。</details><details close><summary> <b>2024-09-01</b> </summary>- 更新minimind-v1 (108M)模型，采用minimind_tokenizer，预训练轮次3 + SFT轮次10，更充分训练，性能更强。- 项目已部署至ModelScope创空间，可以在此网站上体验：- [ModelScope在线体验](https://www.modelscope.cn/studios/gongjy/minimind)</details><details close> <summary> <b>2024-08-27</b> </summary>- 项目首次开源</details># 📌 Environment仅是我个人的软硬件环境配置，自行酌情更改：* Ubuntu == 20.04* Python == 3.9* Pytorch == 2.1.2* CUDA == 12.2* [requirements.txt](./requirements.txt)# 📌 Quick Inference & Test<div align="center" style="font-size: 1.5em; font-weight: bold;"> <img src="https://huggingface.co/front/assets/huggingface_logo-noborder.svg" alt="Hugging Face Logo" style="vertical-align: middle; height: 30px;" /> Hugging Face[MiniMind (HuggingFace)](https://huggingface.co/collections/jingyaogong/minimind-66caf8d999f5c7fa64f399e5) <img src="https://g.alicdn.com/sail-web/maas/1.15.0/static/modelscopeIcon.cd89353f.svg" alt="Hugging Face Logo" style="vertical-align: middle; height: 30px;" />[MiniMind (ModelScope)](https://www.modelscope.cn/models/gongjy/minimind-v1)</div>```bash# step 1git clone https://huggingface.co/jingyaogong/minimind-v1``````bash# step 2python 2-eval.py```或者启动streamlit，启动网页聊天界面```bash# or step 3, use streamlitstreamlit run fast_inference.py```![](./images/streamlit.png)<div align="center">项目已部署至ModelScope创空间，可以在此网站上体验：[ModelScope在线体验](https://www.modelscope.cn/studios/gongjy/minimind)</div># 📌 Quick Start* 0、环境安装 ```bash pip install -r requirements.txt -i https://pypi.tuna.tsinghua.edu.cn/simple ```* 1、克隆项目代码 ```text git clone https://github.com/jingyaogong/minimind.git ```* 2、如果你需要自己训练 * 2.1 下载[数据集下载地址](#数据集下载地址)放到`./dataset`目录下 * 2.2 `python data_process.py`处理数据集，例如pretrain数据提前进行token-encoder、sft数据集抽离qa到csv文件 * 2.3 在`./model/LMConfig.py` 中调整model的参数配置 * 2.4 `python 1-pretrain.py` 执行预训练 * 2.5 `python 3-full_sft.py` 执行指令微调 * 2.6 `python 4-lora_sft.py` 执行lora微调（非必须） * 2.7 `python 5-dpo_train.py` 执行DPO人类偏好强化学习对齐（非必须）* 3、测试模型推理效果 * 确保需要使用的，训练完成的参数权重位于`./out/`目录下 * 也可以直接去[训练完成的模型权重](#训练完成的模型权重)下载使用我训练好的 ```text out ├── multi_chat │ ├── full_sft_512.pth │ ├── full_sft_512_moe.pth │ └── full_sft_768.pth ├── single_chat │ ├── full_sft_512.pth │ ├── full_sft_512_moe.pth │ └── full_sft_768.pth ├── pretrain_768.pth ├── pretrain_512_moe.pth ├── pretrain_512.pth ``` * `python 0-eval_pretrain.py`测试预训练模型的接龙效果 * `python 2-eval.py`测试模型的对话效果 ![2-eval](./images/2-eval.png)🍭 【Tip】预训练和全参微调pretrain和full_sft均支持多卡加速* 单机N卡启动训练(DDP) ```bash torchrun --nproc_per_node N 1-pretrain.py # and torchrun --nproc_per_node N 3-full_sft.py ```* 单机N卡启动训练(DeepSpeed) ```bash deepspeed --master_port 29500 --num_gpus=N 1-pretrain.py # and deepspeed --master_port 29500 --num_gpus=N 3-full_sft.py ```# 📌 Data sources- 🤖 分词器：nlp中的Tokenizer类似于词典，将单词从自然语言通过“词典”映射到0,1,36这样的数字，可以理解为数字就代表了单词在“词典�

用户评论 (0)

发表评论

您的昵称:

评分

评论内容:

验证码:

minimind最小开源大模型，可以自己训练自己的大模型

资源文件列表(大概)

资源内容介绍

用户评论 (0)

发表评论

相关资源

RDTU H071 (2023-11-23 16-57-05).zip

点餐系统微信小程序+SpringBoot毕业设计源码+数据库+论文+启动教程.zip

宿舍管理小程序微信小程序+SpringBoot毕业设计源码+数据库+论文+启动教程.zip

学生宿舍管理系统微信小程序+SpringBoot毕业设计源码+数据库+论文+启动教程.zip

minimind最小开源大模型，可以自己训练自己的大模型

资源文件列表(大概)

资源内容介绍

用户评论 (0)

发表评论

相关资源

RDTU H071 (2023-11-23 16-57-05).zip

点餐系统 微信小程序+SpringBoot毕业设计 源码+数据库+论文+启动教程.zip

宿舍管理小程序 微信小程序+SpringBoot毕业设计 源码+数据库+论文+启动教程.zip

学生宿舍管理系统 微信小程序+SpringBoot毕业设计 源码+数据库+论文+启动教程.zip

点餐系统微信小程序+SpringBoot毕业设计源码+数据库+论文+启动教程.zip

宿舍管理小程序微信小程序+SpringBoot毕业设计源码+数据库+论文+启动教程.zip

学生宿舍管理系统微信小程序+SpringBoot毕业设计源码+数据库+论文+启动教程.zip