How We Measure Trust
Building domain-capable models requires more than fine-tuning. It demands rigorous evaluation, privacy-aware architecture, and disciplined release gates. This is how AstralMaris approaches that work.
No single benchmark captures model quality. We evaluate across multiple lanes.
Privacy and security are release-gate concerns, not legal footnotes.
Retrieval-first patterns are our default for regulated environments.
How We Evaluate Models
A single benchmark score is not a trustworthy measure of model quality. Our evaluation framework layers multiple measurement types to build confidence in what a model actually knows and where it might fail.
Public Benchmark Lane
Standard academic benchmarks establish baseline capability on reasoning, comprehension, and factual recall. We use these for initial screening, not final judgment.
Private Task Benchmarks
Domain-specific tasks designed with the client to measure performance on actual work. These are the ground truth for whether the model thinks in the shape of the domain.
Calibration & Abstention Testing
Models must know the edges of their capability. We test boundary awareness— does the model refuse gracefully when it should, or does it hallucinate with false confidence?
Privacy & Leakage Testing
Adversarial extraction attempts to surface memorized training data. We measure what the model might reveal under pressure.
Operational Monitoring
Post-deployment: latency, cost, error rates, and drift detection. Models degrade. We monitor for it.
Privacy & Regulated Data
The safest default for regulated environments is retrieval-first architecture with strong governance, not unrestricted fine-tuning on sensitive data.
Our Position
- — Minimize data. Only train on what is necessary.
- — De-identify where possible before any adaptation.
- — Avoid fine-tuning on PHI and PII unless governed paths exist.
- — Prefer retrieval for factual knowledge in sensitive domains.
When We Fine-Tune
Fine-tuning on sensitive data requires explicit justification and additional controls:
- → Data provenance documentation
- → Access control and logging
- → Retention and deletion protocols
- → Leakage testing before release
Security & Threat Modeling
Domain-adapted models face distinct threat classes. We model these explicitly and build controls into the development pipeline.
Training Data Poisoning
Malicious or corrupted data inserted into training corpora. Mitigated through provenance tracking and source validation.
Memorization & Leakage
Models regurgitating sensitive training data under adversarial prompting. Tested for and measured before release.
Artifact Tampering
Model weights or configuration modified post-training. Addressed through signing and digest verification.
Prompt Injection
Adversarial inputs that override intended behavior. Mitigated through boundary testing and input validation.
Benchmark Gaming
Training specifically to score well on evaluation metrics without genuine capability gain. Detected through held-out test sets.
Deployment Drift
Model behavior degrading over time as inputs or environment change. Monitored through operational telemetry.
What We Deliver
Evaluation Framework
Documented methodology for measuring model capability, including benchmark selection, acceptance criteria, and human review rubrics.
Release-Gate Discipline
Minimum quality bars for capability, boundary behavior, leakage resistance, and artifact integrity before any model ships.
Privacy-Aware Architecture
Guidance on data handling, de-identification, and architecture selection for regulated environments (PHI, PII, confidential data).
Threat Model & Controls
Documented threat classes, attack vectors, and operational controls for your specific deployment context.
The AstralMaris Method
Our methodology emerged from practice, not theory. Six stages, each with eval-gated checkpoints. The deliverable is not a model—it is capability transfer.
The Survey
Walk the domain with practitioners. Discover tacit knowledge.
Knowledge Extraction
Structure the domain into queryable entity-relationship graphs.
The Knowledge Partition
Separate what belongs in weights from what stays in retrieval.
Sequential Training
SFT → Self-distillation → Quantization-aware distillation. No shortcuts.
The Atelier
Co-teaching with experts. Side-by-side comparison and refinement.
Capability Transfer
Deliver model + methodology + eval suite. Client can retrain independently.
Start With a Technical Conversation
If you're evaluating partners for domain-adapted model work, we should discuss your specific requirements, constraints, and threat model. No audit. No assessment. Just clarity about what would actually help.
For security reviewers: Our full customer assurance library is available under NDA. Include security questionnaire requirements in your initial inquiry.