rasbt/LLMs-from-scratch
1
89.6k
Fork 0
mirror of https://github.com/rasbt/LLMs-from-scratch.git synced 2026-03-31 09:31:44 +00:00
Code Issues Packages Projects Releases Wiki Activity
1,040 Commits 11 Branches 0 Tags
main
Commit Graph

80 Commits

Author SHA1 Message Date
dagecko
6b9502056f fix: pin 1 unpinned action(s) (#987) 
Automated security fixes applied by Runner Guard (https://github.com/Vigilant-LLC/runner-guard).

Changes:
 .github/workflows/basic-tests-pixi.yml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
 2026-03-26 11:49:44 -05:00
Sebastian Raschka
e155d1b02c Update unit tests for CI (#952) 
* Update CI

* Revert submodule pointer update

* Update

* update

* update
 2026-01-27 17:44:55 -06:00
Pádraic Slattery
59d9262047 chore: Update outdated GitHub Actions versions (#951) 2026-01-19 12:22:29 -06:00
Sebastian Raschka
14c7afaa58 Fix GitHub CI timeout issue for link checker (#937) 
* Fix GitHub CI timeout issue for link checker

* update problematic links
 2026-01-02 14:34:31 -06:00
Sebastian Raschka
1c9f49c812 Add some appendix E runtimes (#927) 
* Add some appendix E runtimes

* Use more lightweigth CPU torch

* Use more lightweigth CPU torch

* Use more lightweigth CPU torch
 2025-12-19 14:13:43 -06:00
Sebastian Raschka
bc6f335526 Olmo 3 from scratch (#914) 
* Olmo 3 from scratch

* update

* update

* update
 2025-11-22 22:42:18 -06:00
Aviral Garg
27d52d6378 Fix MHAEinsum weight dimension bug when d_in != d_out (#857) (#893) 
* Fix MHAEinsum weight dimension bug when d_in != d_out (#857)

Previously MHAEinsum initialized weight matrices with shape (d_out, d_in) and used inappropriate einsum notation, causing failures for non-square input-output dimensions. This commit corrects weight initialization to shape (d_in, d_out), updates einsum notation to 'bnd,do->bno', and adds three unit tests to verify parity across different d_in and d_out settings. All tests pass successfully.

* use pytest

* Update .gitignore

---------

Co-authored-by: rasbt <mail@sebastianraschka.com>
 2025-10-31 21:45:31 -05:00
Sebastian Raschka
7ca7c47e4a Make quote style consistent (#891) 2025-10-21 19:42:33 -05:00
Sebastian Raschka
7bd263144e Switch from urllib to requests to improve reliability (#867) 
* Switch from urllib to requests to improve reliability

* Keep ruff linter-specific

* update

* update

* update
 2025-10-07 15:22:59 -05:00
Sebastian Raschka
458f2d9b67 Test dependencies with Python 3.13 (#843) 
* Custom python 3.13 entry in pyproject.toml

* amend

* update

* update

* update

* Update pyproject.toml

* Update pyproject.toml

* Update pyproject.toml

* update
 2025-09-27 08:38:07 -05:00
Sebastian Raschka
18c6b970ab Add additional notes on debugging SSL issues (#810) 
* Add SSL instructions

* update old pytorch tests

* update

* update

* update

* update

* update

* update

* update

* update
 2025-09-06 11:46:50 -05:00
Sebastian Raschka
f571b5e493 Add Gemma3 KV cache variant (#776) 
* Add Gemma3 KV cache variant

* update
 2025-08-19 12:37:49 -05:00
Sebastian Raschka
80d4732456 add HF equivalency tests for standalone nbs (#774) 
* add HF equivalency tests for standalone nbs

* update

* update

* update

* update
 2025-08-18 18:58:46 -05:00
Sebastian Raschka
a6b883c9f9 Gemma 3 270M From Scratch (#771) 
* Gemma 3 270M From Scratch

* fix path

* update readme
 2025-08-17 08:23:05 -05:00
Sebastian Raschka
b14325e56d Qwen3 and Llama3 equivalency teests with HF transformers (#768) 
* Qwen3 and Llama3 equivalency teests with HF transformers

* update
 2025-08-14 18:36:07 -05:00
Sebastian Raschka
f92b40e4ab Qwen3 Coder Flash & MoE from Scratch (#760) 
* Qwen3 Coder Flash & MoE from Scratch

* update

* refinements

* updates

* update

* update

* update
 2025-08-01 19:13:17 -05:00
Matthew Hernandez
6f12edb0cc Fix issue: 731 by resolving semantic error (#738) 
* fix issue 731

* update test path

---------

Co-authored-by: rasbt <mail@sebastianraschka.com>
 2025-07-10 16:30:15 -05:00
Sebastian Raschka
9cf64170ed Update Qwen3 tokenizer test (#727) 
* Update Qwen3 tokenizer test

* add tokenizers to dev dependencies

* add tokenizers to dev dependencies
 2025-07-08 06:59:46 -05:00
Martin Ma
6522be94be Fix bug in masking when kv cache is used. (#697) 
* Fix bug in masking when kv cache is used.

* add tests

* dd tests

* upd

* add kv cache test to gh workflow

* explicit mask slicing

* upd

---------

Co-authored-by: rasbt <mail@sebastianraschka.com>
 2025-06-23 13:12:56 -05:00
Sebastian Raschka
92b308e512 Ch06 and Ch07 videos (#613) 
* Ch06 and Ch07 videos

* exclude google scholar from link checking
 2025-04-12 14:51:02 -05:00
Sebastian Raschka
67e0680210 Disable mask saving as weight in Llama 3 model (#604) 
* Disable mask saving as weight

* update pixi

* update pixi
 2025-04-06 09:33:36 -05:00
Sebastian Raschka
aedad7efc3 Add Llama 3.2 to pkg (#591) 
* Add Llama 3.2 to pkg

* remove redundant attributes

* update tests

* updates

* updates

* updates

* fix link

* fix link
 2025-03-31 18:59:47 -05:00
Sebastian Raschka
c21bfe4a23 Add PyPI package (#576) 
* Add PyPI package

* fixes

* fixes
 2025-03-23 19:28:49 -05:00
Sebastian Raschka
f63f04d8d5 Fix BPE bonus materials (#561) 
* Fix BPE bonus materials

* fix bpe implementation

* update

* Add 'Hello, world. Is this-- a test?' test case

* update link to test file

* update path handling

* update path handling

* fix pytest paths
 2025-03-08 17:21:30 -06:00
Sebastian Raschka
8939fdc846 Add setup video tutorial (#547) 
* Add setup video tutorial

* updated link checks

* updated link checks
 2025-02-26 13:53:22 -06:00
Sebastian Raschka
bca82a9443 Simplify the setup in CI (#542) 
* Simplify the setup in CI

* windows --no-binary option

* windows uv workflow

* add path

* some fixes

* some fixes

* some fixes

* cleanup

* cleanup

* cleanup

* cleanup

* cleanup

* more windows issues

* more windows issues

* more windows issues

* more windows issues

* more windows issues

* more windows issues

* more windows issues

* more windows issues

* revert
 2025-02-20 11:56:52 -06:00
Tim Hopper
1039bf9b80 Don't explicitly install Python. Use uv sync instead of uv add. (#540) 
* Don't explicitly install Python. Use uv sync instead of uv add.

* updates

* update

* updated 3.11 workflows

---------

Co-authored-by: rasbt <mail@sebastianraschka.com>
 2025-02-19 10:42:13 -06:00
rasbt
24f78865df update badges 2025-02-17 12:00:46 -06:00
rasbt
2f67cbca0b update readme badges 2025-02-17 11:49:41 -06:00
Matthew Feickert
a8b8eb4731 feat: Add pixi environment (#534) 
* feat: Add pixi environment

* Add pixi manifest pixi.toml for Linux x86, macOS arm64, Windows 64.

* ci: Update CI workflow and unify to one

* Enable workflow dispatch.
* Add concurrency limits.
* Use pixi for CI workflow.
* Unify to a single workflow for all OS tested

* feat: Add pixi lock file

* Ensure tensorflow-cpu installed on Windows

* fix package check

* fix package check

* simplification plus uv and pip runners

* some fixes to pixi and pip

* create pixi.lock

* fix pixi.lock issue

* another attempt trying to fix get_packages

* another attempt trying to fix get_packages

* clean up python_environment_check.py

* updated runner and docs

* use bash

* proper env activiation

* proper env activiation

---------

Co-authored-by: rasbt <mail@sebastianraschka.com>
 2025-02-17 11:33:53 -06:00
Sebastian Raschka
a08d7aaa84 Uv workflow improvements (#531) 
* Uv workflow improvements

* Uv workflow improvements

* linter improvements

* pytproject.toml fixes

* pytproject.toml fixes

* pytproject.toml fixes

* pytproject.toml fixes

* pytproject.toml fixes

* pytproject.toml fixes

* windows fixes

* windows fixes

* windows fixes

* windows fixes

* windows fixes

* windows fixes

* win32 fix

* win32 fix

* win32 fix

* win32 fix

* win32 fix

* win32 fix

* win32 fix

* win32 fix

* win32 fix

* win32 fix

* win32 fix

* win32 fix

* win32 fix

* win32 fix

* win32 fix

* win32 fix

* win32 fix

* win32 fix

* win32 fix
 2025-02-16 13:16:51 -06:00
Sebastian Raschka
88fd849b88 Switch from pip to uv (#529) 
* Replace pip by more modern uv

* uv tests

* update yaml

* update yaml

* update yaml

* update flake8

* update windows commands

* fix windows test

* windows fix

* windows fix

* windows fix

* windows fix

* windows fix

* windows fix

* windows fix

* windows fix

* windows fix

* windows fix
 2025-02-15 13:13:13 -06:00
Sebastian Raschka
a6cc574605 Upgrade to NumPy 2.0 (#520) 
* Upgrade to NumPy 2.0

* bump pytorch

* bump pytorch

* bump pytorch

* bump pytorch

* bump pytorch

* update

* update packages
 2025-02-09 06:21:58 -06:00
Sebastian Raschka
68e2efe1c9 Mention small discrepancy due to Dropout non-reproducibility in PyTorch (#519) 
* Mention small discrepancy due to Dropout non-reproducibility in PyTorch

* bump pytorch version
 2025-02-06 14:59:52 -06:00
Sebastian Raschka
0e14c76dee Test PyTorch nightly releases (#509) 2025-01-30 12:45:48 -06:00
Sebastian Raschka
0911e71497 Test for PyTorch 2.6 release candidate (#500) 
* Test for PyTorch 2.6 release candidate

* update

* update

* remove extra added file
 2025-01-22 18:37:48 -06:00
Sebastian Raschka
ccade77bf4 Add flexible padding bonus experiment (#438) 
* Add flexible padding bonus experiment

* fix links
 2024-11-15 08:51:01 +09:00
Sebastian Raschka
467197bbf5 Test PyTorch 2.5 (#405) 2024-10-20 10:23:31 -05:00
Sebastian Raschka
233a3b0c8b Update check-links.yml 2024-10-11 12:20:49 -05:00
Sebastian Raschka
1a8d2929dd Update check-links.yml 2024-10-08 08:38:48 -05:00
Sebastian Raschka
8a448a4410 Llama 3 (#384) 
* Implement Llama 3.2

* Add Llama 3.2 files

* exclude IMDB link because stanford website seems down
 2024-10-05 07:52:15 -05:00
Sebastian Raschka
b56d0b2942 Add llama2 unit tests (#372) 
* add llama2 unit tests

* update

* updates

* updates

* update file path

* update requirements file

* rmsnorm test

* update
 2024-09-25 19:40:36 -05:00
rasbt
b4f72812ed update link checking 2024-09-15 08:16:23 -05:00
Sebastian Raschka
e130ca293c Update matplotlib tests on Windows (#295) 2024-08-04 09:18:19 -05:00
Sebastian Raschka
263eee8921 Test with PyTorch 2.0 and 2.4 (#290) 
* Test with PyTorch 2.0 and 2.4

* Update basic-tests-old-pytorch.yml

* skip version cell
 2024-07-27 15:09:02 -05:00
Sebastian Raschka
45d0318ae6 Gh workflow for spelling errors (#265) 
* Gh workflow for spelling errors

* add files to check
 2024-07-16 08:17:20 -05:00
Daniel Kleine
1db1999951 minor fixes (#248) 
* removed duplicated numpy req

* labels to array

* autopep8 NBs
 2024-06-26 08:52:51 -05:00
Sebastian Raschka
6c0dc2362b Add standalone finetuning and evaluation scripts for chapter 7 (#234) 
* add finetuning and eval scripts

* update link

* update links

* fix link
 2024-06-21 05:23:24 -05:00
Sebastian Raschka
24523bd34d Add pytest retry for link checks (#228) 
* add pytest retry for link checks

* Update .github/workflows/check-links.yml

* newline
 2024-06-19 07:37:41 -05:00
Daniel Kleine
fba205985d minor fixes (#222) 
* fixed labels

* fixed typo

---------

Co-authored-by: Sebastian Raschka <mail@sebastianraschka.com>
 2024-06-18 19:37:26 -05:00