Design, develop, and optimize Small Language Models for healthcare and cybersecurity domains using advanced architectures (efficient transformers, MoE, sparse attention), compression techniques (quantization: INT8/INT4/GPTQ/AWQ, pruning, knowledge distillation), and emerging SLM architectures (Phi-3, Gemma, Mistral 7B variants) with rigorous ablation studies balancing model size, inference speed, accuracy, and resource requirements 

Implement parameter-efficient fine-tuning (PEFT) techniques including LoRA, QLoRA, prefix tuning, prompt tuning, and adapter layers; design instruction tuning pipelines aligning SLMs with clinical terminology and security protocols; develop multi-task learning frameworks; and deploy continuous learning strategies using DPO, RLHF, and active learning while optimizing hyperparameters to prevent overfitting on limited healthcare datasets 

Implement knowledge graphs capturing domain relationships and build graph-augmented language models integrating Neo4j/Cosmos DB with unstructured text processing, leveraging GNN components (PyTorch Geometric, DGL, GraphSAGE) for entity relationship modeling and complex graph traversal reasoning 

Create entity extraction and relation classification pipelines that automatically populate and maintain knowledge graphs from clinical documents, security logs, and operational data to enable graph-based reasoning across patient journeys, care coordination workflows, and security incident chains 

Optimize model inference using vLLM, TensorRT-LLM, llama.cpp, GGML, and Optimum for maximum throughput/minimum latency; implement scalable model serving architectures (Triton, TorchServe, FastAPI) with batching, caching, and load balancing; design A/B testing frameworks and champion-challenger patterns for safe production model updates 

Design and implement vector database architectures for semantic search and RAG applications using specialized embeddings optimized for cyber text, with hybrid search combining dense vector retrieval, sparse BM25 matching, and graph-based context expansion 

Build advanced RAG architectures implementing multi-vector retrieval, reranking strategies (Cohere, Cross-Encoders), context compression, and query decomposition; optimize vector indexing (HNSW, IVF, Product Quantization) balancing accuracy, latency, and memory footprint; and develop chunking pipelines for diverse cyber data formats 

Experience in end-to-end data pipelines for cyber text corpora processing, implementing data quality frameworks through de-identification and synthetic data generation (Synthea), bias detection, and representativeness metrics while building experiment tracking systems (MLflow) for versioning and reproducibility 

Develop distributed training workflows using DeepSpeed, FSDP, or Megatron-LM for efficient multi-GPU/multi-node training on Azure ML; create automated data augmentation pipelines (back-translation, paraphrasing, entity substitution, synthetic generation) to expand limited healthcare training datasets while maintaining clinical validity 

Design comprehensive evaluation frameworks measuring healthcare-specific task performance (NER, relation extraction, summarization, QA, reasoning) with custom benchmark datasets aligned to clinical accuracy requirements, regulatory compliance, and operational impact; implement automated robustness testing for adversarial attacks, edge cases, OOD detection, and hallucination detection 

Conduct comparative analysis of SLM variants across accuracy, latency, throughput, memory, and cost metrics; build interpretability tools using attention visualization, SHAP values, and integrated gradients to ensure clinical safety and explainability for healthcare stakeholders and regulatory requirements 

Understand on monitoring and observability dashboards tracking model performance drift, data distribution shifts, embedding quality degradation, and retrieval accuracy in production; collaborate with MLOps teams on containerization (Docker), Kubernetes/AKS deployment, and CI/CD pipelines for automated testing and deployment workflows