Mcore Run On Slurm

NVIDIA/Megatron-LM/skills/mcore-run-on-slurm

Multi-file

NVIDIA· v1.0.0

devopsOfficial

Official Provider SkillView repo

How to launch distributed Megatron-LM training jobs on a SLURM cluster. Covers a minimal sbatch skeleton, environment-variable setup for torch.distributed.run, CUDA_DEVICE_MAX_CONNECTIONS rules across hardware and parallelism modes, container conventions, monitoring, and per-rank failure diagnosis.

Files5 files

BENCHMARK.md65 lines

Loading editor…

Install

Recommended

One command — your agent picks it up automatically.

Select AI agent

Select an AI agent above to see the install command.

Manual Install

More steps

Download the archive and add the files to your project manually.

Skill details

Versionv1.0.0

AuthorNVIDIA

Categorydevops

Skill IDNVIDIA/Megatron-LM/skills/mcore-run-on-slurm

Files5 files

Related skills

Mcore Build And DependencyContainer-based dev environment setup and dependency management for Megatron-LM. Covers acquiring and launching the CI container, uv package management, and updating uv.lock.Mcore Bump Base ImageBump the NVIDIA PyTorch base image (`nvcr.io/nvidia/pytorch:YY.MM-py3`) used by Megatron-LM CI. Covers the two pin sites (GitHub CI in `docker/.ngc_version.dev` and GitLab CI in `.gitlab/stages/01.build.yml`), the post-bump CI loop (re-run functional tests, refresh golden values, mark broken tests), and the gotchas that bit PRs Mcore CicdCI/CD reference for Megatron-LM. Covers CI pipeline structure, PR scope labels, triggering internal GitLab CI (which force-pushes the current branch to a pull-request/BRANCH ref — always dry-run and verify the destination first; never run against shared or protected branches), and CI failure investigation.Mcore Create IssueInvestigate a failing GitHub Actions run or job and create a GitHub issue for the failure.Mcore Linting And FormattingLinting and formatting for Megatron-LM. Covers running autoformat.sh, tools (ruff, black, isort, pylint, mypy), and code style rules.Mcore Onboard Gb200 1node TestsOnboard 1-node GitHub MR functional tests for GB200 from existing mr-scoped 2-node tests.