Commit Graph

1026 Commits

Author SHA1 Message Date
thomas chaton
5a0d2eff8c map operator: Add support for non absolute input_dir and output_dir (#19378) 2024-02-01 08:25:47 +00:00
Jirka Borovec
9d35c61f5f ci: adding missing requirements for generating legacy ckpt (#19353) 2024-01-28 11:22:07 +01:00
awaelchli
1a59097ab2 Drop support for PyTorch 1.12 (#19300)
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
2024-01-26 11:44:24 -05:00
Jirka Borovec
3bd133b107 CI: enable testing with coming PT 2.2 (#19289)
* ci: build dockers for PT 2.2
* py3.12
* --pre --extra-index-url
* typing-extensions
* bump jsonargparse
* install latest jsonargparse
* Add windows skips for Fabric
* convert to xfail
* add pytorch skips
* skip checkpoint consolidation test
* set max torch

---------

Co-authored-by: awaelchli <aedu.waelchli@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2024-01-26 16:42:09 +01:00
thomas chaton
ed367ca675 StreamingDataLoader: Resolve fault tolerance with the CombinedStreamingDataset and multiple workers (#19326) 2024-01-23 17:54:10 +00:00
Carlos Mocholí
a1dd9efcf7 Drop XLA XRT support (#19232)
* Drop XLA XRT support
* update test
* set launched
* update conftest
* xla available check
---------

Co-authored-by: awaelchli <aedu.waelchli@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2024-01-10 18:39:20 +01:00
dependabot[bot]
ae1f78cb9c Bump actions/setup-python from 4 to 5 (#19136)
Bumps [actions/setup-python](https://github.com/actions/setup-python) from 4 to 5.
- [Release notes](https://github.com/actions/setup-python/releases)
- [Commits](https://github.com/actions/setup-python/compare/v4...v5)

---
updated-dependencies:
- dependency-name: actions/setup-python
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
2024-01-09 08:33:09 -05:00
dependabot[bot]
5dbfcc278e Bump Lightning-AI/utilities from 0.10.0 to 0.10.1 (#19211)
Bumps [Lightning-AI/utilities](https://github.com/lightning-ai/utilities) from 0.10.0 to 0.10.1.
- [Release notes](https://github.com/lightning-ai/utilities/releases)
- [Changelog](https://github.com/Lightning-AI/utilities/blob/main/CHANGELOG.md)
- [Commits](https://github.com/lightning-ai/utilities/compare/v0.10.0...v0.10.1)

---
updated-dependencies:
- dependency-name: Lightning-AI/utilities
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-12-26 02:49:11 +01:00
Jirka Borovec
6ea0e2d632 ci: cancel-in-progress if PR (#19204) 2023-12-21 17:49:18 -05:00
Jirka Borovec
02ad844a17 ci: bump also conda requirements (#19203)
* ci: bump also conda requirements
* Apply suggestions from code review

---------

Co-authored-by: Justus Schock <12886177+justusschock@users.noreply.github.com>
2023-12-21 22:58:43 +01:00
Carlos Mocholí
58b4bfab2e Fix standalone test file links (#19165) 2023-12-14 17:18:47 -05:00
Jirka Borovec
3c1dc71447 ci: freeze requirements used in workflows (#19146) 2023-12-14 14:49:43 +01:00
dependabot[bot]
2e77862c24 Bump google-github-actions/setup-gcloud from 1 to 2 (#19137)
Bumps [google-github-actions/setup-gcloud](https://github.com/google-github-actions/setup-gcloud) from 1 to 2.
- [Release notes](https://github.com/google-github-actions/setup-gcloud/releases)
- [Changelog](https://github.com/google-github-actions/setup-gcloud/blob/main/CHANGELOG.md)
- [Commits](https://github.com/google-github-actions/setup-gcloud/compare/v1...v2)

---
updated-dependencies:
- dependency-name: google-github-actions/setup-gcloud
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-12-13 13:47:18 -05:00
dependabot[bot]
bb4421327a Bump google-github-actions/auth from 1 to 2 (#19105)
Bumps [google-github-actions/auth](https://github.com/google-github-actions/auth) from 1 to 2.
- [Release notes](https://github.com/google-github-actions/auth/releases)
- [Changelog](https://github.com/google-github-actions/auth/blob/main/CHANGELOG.md)
- [Commits](https://github.com/google-github-actions/auth/compare/v1...v2)

---
updated-dependencies:
- dependency-name: google-github-actions/auth
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-12-13 13:40:14 -05:00
Jirka Borovec
665c94cef8 ci: update labeler config for v5 (#19151) 2023-12-13 15:00:41 +01:00
dependabot[bot]
1e3b0e7057 Bump actions/labeler from 4 to 5 (#19135)
Bumps [actions/labeler](https://github.com/actions/labeler) from 4 to 5.
- [Release notes](https://github.com/actions/labeler/releases)
- [Commits](https://github.com/actions/labeler/compare/v4...v5)

---
updated-dependencies:
- dependency-name: actions/labeler
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-12-13 09:11:01 +01:00
dependabot[bot]
360b12434e Bump redhat-plumbers-in-action/advanced-issue-labeler from 2 to 3 (#19104)
Bumps [redhat-plumbers-in-action/advanced-issue-labeler](https://github.com/redhat-plumbers-in-action/advanced-issue-labeler) from 2 to 3.
- [Release notes](https://github.com/redhat-plumbers-in-action/advanced-issue-labeler/releases)
- [Commits](https://github.com/redhat-plumbers-in-action/advanced-issue-labeler/compare/v2...v3)

---
updated-dependencies:
- dependency-name: redhat-plumbers-in-action/advanced-issue-labeler
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-12-13 07:35:17 +01:00
Jirka Borovec
5280f9b80e ci: run TPU by label update (#19145)
* ci: run TPU by label again

* labeled

* drop edited

Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>

---------

Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
2023-12-12 14:58:54 -05:00
Jirka Borovec
004b0d1f24 ci: run TPU only with label (#19143)
* ci: run TPU only with label

* drop false

* keep it failing as Carlos likes :)
2023-12-12 12:37:20 -06:00
dependabot[bot]
e7d7399da3 Bump actions/setup-node from 3 to 4 (#18889)
Bumps [actions/setup-node](https://github.com/actions/setup-node) from 3 to 4.
- [Release notes](https://github.com/actions/setup-node/releases)
- [Commits](https://github.com/actions/setup-node/compare/v3...v4)

---
updated-dependencies:
- dependency-name: actions/setup-node
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
2023-12-04 18:39:40 +01:00
Adrian Wälchli
197b22586a Fix comm initialization in MPIEnvironment (#19074) 2023-11-28 16:14:46 -05:00
dependabot[bot]
f5cbd44a04 Bump AButler/upload-release-assets from 2.0 to 3.0 (#19069)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-11-27 12:38:37 +01:00
Adrian Wälchli
da79480054 PyTest random order for Fabric tests (#19040) 2023-11-22 16:41:49 -05:00
thomas chaton
d3df1273b6 Add disk usage check before downloading files (#19041)
Co-authored-by: thomas <thomas@thomass-MacBook-Pro.local>
2023-11-21 20:10:18 +00:00
dependabot[bot]
4789905880 Bump Lightning-AI/utilities from 0.9.0 to 0.10.0 (#19031)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-11-20 05:52:30 -05:00
Carlos Mocholí
4e72dcc8db Reduce lightning data's dependencies (#19026) 2023-11-17 16:52:14 -05:00
Jirka Borovec
b80107e7ba ci/docs: upload with dispatch (#19018)
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2023-11-16 21:08:10 +01:00
Jirka Borovec
beca90933c docs: enable retro dispatch build (#19016) 2023-11-16 12:14:42 -05:00
Jirka Borovec
ca7368c5e1 ci/doc: enable build docs on spec version (#19013) 2023-11-16 17:40:13 +01:00
Jirka Borovec
eedff1256c ci: minor formatting update (#19008) 2023-11-15 23:15:01 -05:00
Jirka Borovec
ed76bb74c6 ci: add labels for data & store (#18961) 2023-11-14 20:39:01 +01:00
thomas chaton
6a0f992d53 Improve s3 client support (#18920)
* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* Update src/lightning/data/streaming/client.py

Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>

* update

* update

* update

* update

* update

* update

* update

* update

* update

---------

Co-authored-by: thomas <thomas@thomass-MacBook-Pro.local>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
2023-11-02 13:10:25 -04:00
Jirka Borovec
1e68c50141 ci: restrict build docs on PR (#18923)
* ci: restrict build docs on PR

* linkcheck_timeout
2023-11-02 09:03:03 -04:00
thomas chaton
0843041d1d Add broadcast to Dataset Optimizer with multiple nodes (#18860)
Co-authored-by: Luca Antiga <luca.antiga@gmail.com>
Co-authored-by: thomas <thomas@thomass-MacBook-Pro.local>
2023-10-27 00:42:46 +01:00
Adrian Wälchli
1210deb5a0 Update bug report template for 2.1 (#18816)
Co-authored-by: Jirka <jirka.borovec@seznam.cz>
2023-10-24 12:46:53 -04:00
dependabot[bot]
73f5df0a0a Bump torch from 2.0.1 to 2.1.0 in /requirements (#18752)
Co-authored-by: Jirka <jirka.borovec@seznam.cz>
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
2023-10-24 14:11:54 +02:00
Jirka Borovec
1a5718aa38 ci: fix typo in SHA ref (#18837) 2023-10-23 17:00:00 +02:00
thomas chaton
c68ff6482f Add support for text (#18807)
* update

* update

* update

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

---------

Co-authored-by: thomas <thomas@thomass-MacBook-Pro.local>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2023-10-18 20:28:12 -04:00
PL Ghost
23064294c4 docs: update ref to latest tutorials & fix CI trigger (#18787)
Co-authored-by: Borda <Borda@users.noreply.github.com>
Co-authored-by: Jirka <jirka.borovec@seznam.cz>
Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2023-10-13 17:50:01 +02:00
Jirka Borovec
a2377e9d87 ci/docs: create PR only if needed (#18790) 2023-10-13 16:04:44 +02:00
Jirka Borovec
367b57ee3e docs: fix pages on PyPI (#18792) 2023-10-13 15:58:31 +02:00
Jirka Borovec
a001779b17 ci: simplify/unify make docs targets (#18795) 2023-10-13 15:15:07 +02:00
Jirka Borovec
935616fe9f ci/release: create a PR for release bump (#18791)
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2023-10-12 18:12:21 +02:00
Jirka Borovec
20ce3aeeb4 docs: setting cron for periodical update tutorials (#18783) 2023-10-12 10:25:35 +02:00
Carlos Mocholí
9f17324119 Update probot-check-group.yml to v5.4 (#18782) 2023-10-11 18:10:02 +02:00
Jirka Borovec
4ad8bdbe20 releasing 2.1.0 rc1 (#18762) 2023-10-10 10:14:58 +02:00
Jirka Borovec
0e04760ebe docs: pre-install lai sphinx theme (#18764) 2023-10-09 17:20:03 +02:00
thomas chaton
1d5851ffe2 Introduce Cache 1/n (#18642)
Co-authored-by: Ethan Harris <ethanwharris@gmail.com>
Co-authored-by: Luca Antiga <luca.antiga@gmail.com>
Co-authored-by: thomas <thomas@thomass-MacBook-Pro.local>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2023-10-09 16:06:32 +01:00
Adrian Wälchli
77eef8aff5 Update GPU CI and docker images for PyTorch 2.1 (#18719)
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
Co-authored-by: Jirka <jirka.borovec@seznam.cz>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2023-10-06 08:12:37 -04:00
Adrian Wälchli
c514f1cbea Enable PyTorch 2.1 (#18718) 2023-10-06 07:17:03 -04:00