Self Hosted Llm Batch Inference
awslabs/startups/solution-architecture/plugins/aws-startups-solution-architecture/skills/self-hosted-llm-batch-inference
ai-mlResmi
Resmi Sağlayıcı Skill'iView repo
This skill should be used when a startup without a dedicated ML platform team needs recurring, latency-tolerant offline inference for a self-hosted 1B to 8B generative language model on AWS while minimizing GPU spend and recovering automatically from Spot interruption. It prescribes AWS Batch backed by Amazon ECS Managed Instances Spot, with model artifacts staged in Amazon S3, deterministic input shards, vLLM offline inference on one GPU, bounded retries, and output reconciliation before succes
Dosyalar4 dosya
SKILL.md88 satır
Loading editor…
Kurulum
ÖnerilenTek komut — ajanınız otomatik olarak devreye alır.
Kurulum komutunu görmek için yukarıdan bir AI aracı seçin.
veya
Manuel Kurulum
Daha fazla adımArşivi indirin ve dosyaları projenize manuel olarak ekleyin.
Skill detayları
Versiyonv1.0.0
YazarAWS Labs
Kategoriai-ml
Skill IDawslabs/startups/solution-architecture/plugins/aws-startups-solution-architecture/skills/self-hosted-llm-batch-inference
Dosyalar4 dosya
İlgili skill'ler
Agent AdvisorUnified entry point for AI-agent work on AWS: evaluate and pick a runtime, generate a full migration plan (for existing workloads), and build an executable POC — all in one flow. Triggers on: which runtime for my agent, AgentCore vs ECS vs EKS vs Lambda, AgentCore vs Lambda MicroVMs, deploy an AI agent on AWS, agent architecture on AWS, I have an agent idea what do I build, move my agents to AWS, migrate my agents to AWS with a plan, agent migration plan, add AgentCore services, add memory/gatewAgent AdvisorUnified entry point for AI-agent work on AWS: evaluate and pick a runtime, generate a full migration plan (for existing workloads), and build an executable POC — all in one flow. Triggers on: which runtime for my agent, AgentCore vs ECS vs EKS vs Lambda, AgentCore vs Lambda MicroVMs, deploy an AI agent on AWS, agent architecture on AWS, I have an agent idea what do I build, move my agents to AWS, migrate my agents to AWS with a plan, agent migration plan, add AgentCore services, add memory/gatewAgentcore PatternsThis skill should be used when an automated agent's output is published under its own identity and carries weight in someone else's work, such as a reviewer whose verdict lands on a pull request, and the open question is how much authority it has actually earned. Covers which conclusions such an agent may state as settled and which it must hand to a person, why a verdict that moves between runs on unchanged input cannot be allowed to decide anything, handing an uncertain item to a human rather tArchitect For StartupsStartup-tailored AWS architecture advice that adjusts recommendations to the company's stage (pre-revenue through Series B+), team size, runway, and available credits. Use when a founder wants guidance or a recommendation rather than code changes: which services to choose, how to plan or review an architecture, how to stretch credits and control cost, or how to prepare architecture for a fundraise or technical diligence. For an interactive discovery flow that scaffolds and writes the architecturAzure To AwsMigrate workloads from Microsoft Azure to AWS. Triggers on: migrate from Azure, Azure to AWS, move off Azure, migrate Azure app to AWS, migrate AKS to EKS, migrate App Service to AWS, migrate App Service to Elastic Beanstalk, migrate Azure VMs to EC2, migrate Azure SQL to RDS, migrate Azure Database for PostgreSQL to RDS, migrate Azure Database for MySQL to RDS, migrate Cosmos DB to DynamoDB, migrate Azure Cache for Redis to ElastiCache, migrate Blob Storage to S3, migrate Azure Functions to LamAzure To AwsMigrate workloads from Microsoft Azure to AWS. Triggers on: migrate from Azure, Azure to AWS, move off Azure, migrate Azure app to AWS, migrate AKS to EKS, migrate App Service to AWS, migrate App Service to Elastic Beanstalk, migrate Azure VMs to EC2, migrate Azure SQL to RDS, migrate Azure Database for PostgreSQL to RDS, migrate Azure Database for MySQL to RDS, migrate Cosmos DB to DynamoDB, migrate Azure Cache for Redis to ElastiCache, migrate Blob Storage to S3, migrate Azure Functions to Lam