eXtensible Forward Time SIMulator
xftsim simulates complex phenotype/genotype data with an emphasis on short timescale phenomena. xftsim is designed with two primary goals:
make it easy for statistical geneticists to perform reproducible and systematic sensitivity analyses to better understand limitations and assumptions
enable evaulation of methods for analyzing complex traits under realistically complex generative models
Getting started
User guide
- Nuts and bolts
- Anatomy of a simulation
- Founder data
- Recombination maps
- Phenogenetic architectures
- Mating regimes
- Statistics
- Post-processing
- Advanced genetic architectures
Guides (New)
API Reference (New)
Example Notebooks
- Simple Simulation
- 1. Imports
- 2. Create Founder Haplotypes
- 3. Define Genetic Effects
- 4. Define the Architecture via Formula DSL
- 5. Configure Mating and Recombination
- 6. Set Up and Run the Simulation
- 7. Access Phenotype History
- 8. Access Haplotype History
- 9. Inspect Statistics Results
- 10. Variance Decomposition
- Summary
- Bivariate Assortative Mating
- 1. Imports
- 2. Create Founder Haplotypes
- 3. Define Multivariate Genetic Effects
- 4. Define the Bivariate Architecture
- 5. Configure Assortative Mating
- 6. Run Simulation with Assortative Mating
- 7. Run a Control Simulation with Random Mating
- 8. Track Variance Over Generations
- 9. Spousal Correlation
- 10. Full Covariance Matrix at Final Generation
- Summary
- Vertical Transmission
- 1. Imports
- 2. Create Founder Haplotypes
- 3. Define the Architecture with Vertical Transmission
- 4. Define a Control Architecture (No VT)
- 5. Run Simulations with TrioFilter
- 6. Variance Components Over Generations
- 7. Extract Trio Views
- 8. Understanding the VT Inflation
- 9. Genetic vs Total Parent-Offspring Regression
- 10. Continue the Simulation
- Summary
- GWAS and PGS
- 1. Imports
- 2. Run a Simulation with Known Genetic Architecture
- 3. Run GWAS on the Latest Generation
- 4. Manhattan-Style Output
- 5. Beta Recovery: True vs Estimated Effects
- 6. Compute PGS with True Weights
- 7. Compute PGS with GWAS-Estimated Weights
- 8. Using UnrelatedFilter for GWAS
- 9. Out-of-Sample PGS Evaluation
- 10. Running GWAS on All Phenotype Components
- Summary
- Checkpoint and Resume
- Sibling Effects
- 1. Imports
- 2. Define Architecture with Sibling Mean Effect
- 3. Define a Control Architecture (No Sibling Effect)
- 4. Run Simulations
- 5. Compare Sibling Correlations
- 6. Extract Sibling Pairs with SibPairFilter
- 7. Haseman-Elston Estimator Results
- 8. Other Sibling Aggregation Functions
- 9. Variance Decomposition with Sibling Effect
- Summary
- GRG Genotypes
- 1. Imports and pygrgl Availability Check
- 2. Locate GRG Fixture Files
- 3. Loading a GRG File
- 4. Inspecting the GraphHaplotypeOperator
- 5. Matrix-Vector Operations
- 6. Dense Materialization and Subsetting
- 7. Running a Simulation with GRG-Backed Haplotypes
- 8. Verifying GRG vs Dense Simulation Equivalence
- 9. Current Limitations and Future Directions
- Summary
API reference (Legacy)
- Submodule organization
- arch module
ArchComponentGeneticComponentMVGeneticComponentHaplotypeGeneticComponentNoiseComponentCNoiseComponentThresholdComponentAggregationComponentMotherComponentFatherComponentParentComponentSiblingMeanComponentSiblingSumComponentSiblingAnyComponentSiblingCountComponentSiblingEldestComponentSiblingYoungestComponentArchNodeBUILTINSArchitecture
- effect module
- founders module
- index module
XftIndexSampleIndexDiploidVariantIndexDiploidVariantIndex.vidDiploidVariantIndex.chromDiploidVariantIndex.zero_alleleDiploidVariantIndex.one_alleleDiploidVariantIndex.hcopyDiploidVariantIndex.afDiploidVariantIndex.pos_bpDiploidVariantIndex.pos_cMDiploidVariantIndex.ploidyDiploidVariantIndex.annotationDiploidVariantIndex.annotation_arrayDiploidVariantIndex.annotation_namesDiploidVariantIndex.mDiploidVariantIndex.n_chromDiploidVariantIndex.n_annotationsDiploidVariantIndex.mafDiploidVariantIndex.vidDiploidVariantIndex.chromDiploidVariantIndex.one_alleleDiploidVariantIndex.zero_alleleDiploidVariantIndex.hcopyDiploidVariantIndex.afDiploidVariantIndex.pos_bpDiploidVariantIndex.pos_cMDiploidVariantIndex.ploidyDiploidVariantIndex.annotationDiploidVariantIndex.annotation_arrayDiploidVariantIndex.annotation_namesDiploidVariantIndex.mDiploidVariantIndex.n_chromDiploidVariantIndex.n_annotationsDiploidVariantIndex.mafDiploidVariantIndex.to_haploid()DiploidVariantIndex.annotate()
HaploidVariantIndexHaploidVariantIndex.vidHaploidVariantIndex.chromHaploidVariantIndex.zero_alleleHaploidVariantIndex.one_alleleHaploidVariantIndex.afHaploidVariantIndex.pos_bpHaploidVariantIndex.pos_cMHaploidVariantIndex.mHaploidVariantIndex.n_chromHaploidVariantIndex.n_annotationsHaploidVariantIndex.mafHaploidVariantIndex.ploidyHaploidVariantIndex.hcopyHaploidVariantIndex.to_diploid()HaploidVariantIndex.ploidyHaploidVariantIndex.to_diploid()
ComponentIndexComponentIndex.phenotype_nameComponentIndex.component_nameComponentIndex.vorigin_relativeComponentIndex.k_totalComponentIndex.k_phenotypesComponentIndex.k_componentsComponentIndex.k_relativeComponentIndex.depthComponentIndex.unique_identifierComponentIndex.to_vorigin()ComponentIndex.to_proband()ComponentIndex.from_frame()ComponentIndex.from_arrays()ComponentIndex.from_product()ComponentIndex.range_index()ComponentIndex.to_vorigin()ComponentIndex.to_haploid()ComponentIndex.to_proband()ComponentIndex.phenotype_nameComponentIndex.unique_identifierComponentIndex.comp_typeComponentIndex.component_nameComponentIndex.vorigin_relativeComponentIndex.k_totalComponentIndex.k_phenotypesComponentIndex.k_componentsComponentIndex.k_relativeComponentIndex.depthComponentIndex.from_frame()ComponentIndex.from_arrays()ComponentIndex.from_product()ComponentIndex.range_index()
sampleIndex_from_plink()sampleIndex_from_VCF()variantIndex_from_plink()variantIndex_from_VCF()SampleFilterNullFilterRandomSiblingFilterRandomSubsampleFilterRandomSiblingSubsampleFilterSiblingPairFilter
- io module
- Public API
genotypes_to_pseudo_haplotypes()read_plink1_as_pseudohaplotypes()haplotypes_from_sgkit_dataset()save_haplotypes_npz()load_haplotypes_npz()load_grg()save_phenotypes_npz()load_phenotypes_npz()save_effects_npz()load_effects_npz()save_architecture()load_architecture()save_simulation_checkpoint()load_simulation_checkpoint()plink1_variant_index()plink1_sample_index()
- mate module
- ped module
- reproduce module
- sim module
- stats module
- struct module
GeneticMapSampleMetaSampleMeta.nSampleMeta.n_famSampleMeta.n_femaleSampleMeta.n_maleSampleMeta.iidSampleMeta.fidSampleMeta.sexSampleMeta.generationSampleMeta.extraSampleMeta.nSampleMeta.n_famSampleMeta.n_femaleSampleMeta.n_maleSampleMeta.unique_identifierSampleMeta.subset()SampleMeta.with_generation()SampleMeta.to_sample_index()
VariantMetaHaplotypeOperatorHaplotypeOperator.samplesHaplotypeOperator.variantsHaplotypeOperator.nHaplotypeOperator.mHaplotypeOperator.matvec()HaplotypeOperator.rmatvec()HaplotypeOperator.matvec_maternal()HaplotypeOperator.matvec_paternal()HaplotypeOperator.standardized_matvec()HaplotypeOperator.recompute_af()HaplotypeOperator.to_dense()HaplotypeOperator.meiosis()
DenseHaplotypeArrayDenseHaplotypeArray.generationDenseHaplotypeArray.nDenseHaplotypeArray.mDenseHaplotypeArray.diploid_genotypesDenseHaplotypeArray.iidDenseHaplotypeArray.fidDenseHaplotypeArray.sexDenseHaplotypeArray.n_famDenseHaplotypeArray.n_femaleDenseHaplotypeArray.n_maleDenseHaplotypeArray.vidDenseHaplotypeArray.chromDenseHaplotypeArray.pos_bpDenseHaplotypeArray.pos_cMDenseHaplotypeArray.afDenseHaplotypeArray.matvec()DenseHaplotypeArray.rmatvec()DenseHaplotypeArray.standardized_matvec()DenseHaplotypeArray.standardized_rmatvec()DenseHaplotypeArray.diploid_matvec()DenseHaplotypeArray.standardized_haploid_matvec()DenseHaplotypeArray.subset()DenseHaplotypeArray.drop_isel()DenseHaplotypeArray.shapeDenseHaplotypeArray.dataDenseHaplotypeArray.valuesDenseHaplotypeArray.attrsDenseHaplotypeArray.af_empiricalDenseHaplotypeArray.to_diploid_standardized()DenseHaplotypeArray.get_sample_indexer()DenseHaplotypeArray.get_variant_indexer()DenseHaplotypeArray.matvec_maternal()DenseHaplotypeArray.matvec_paternal()DenseHaplotypeArray.recompute_af()DenseHaplotypeArray.to_dense()DenseHaplotypeArray.meiosis()DenseHaplotypeArray.xft
GraphHaplotypeOperatorGraphHaplotypeOperator.nGraphHaplotypeOperator.mGraphHaplotypeOperator.generationGraphHaplotypeOperator.matvec()GraphHaplotypeOperator.rmatvec()GraphHaplotypeOperator.matvec_maternal()GraphHaplotypeOperator.matvec_paternal()GraphHaplotypeOperator.standardized_matvec()GraphHaplotypeOperator.recompute_af()GraphHaplotypeOperator.to_dense()GraphHaplotypeOperator.meiosis()
StandardizedHaplotypeOperatorStandardizedHaplotypeOperator.samplesStandardizedHaplotypeOperator.variantsStandardizedHaplotypeOperator.nStandardizedHaplotypeOperator.mStandardizedHaplotypeOperator.meansStandardizedHaplotypeOperator.stdsStandardizedHaplotypeOperator.matvec()StandardizedHaplotypeOperator.rmatvec()StandardizedHaplotypeOperator.standardized_matvec()StandardizedHaplotypeOperator.matvec_maternal()StandardizedHaplotypeOperator.matvec_paternal()StandardizedHaplotypeOperator.recompute_af()StandardizedHaplotypeOperator.to_dense()StandardizedHaplotypeOperator.meiosis()
HaplotypeArrayAccessorHaplotypeArrayAccessor.nHaplotypeArrayAccessor.mHaplotypeArrayAccessor.generationHaplotypeArrayAccessor.samplesHaplotypeArrayAccessor.variantsHaplotypeArrayAccessor.af_empiricalHaplotypeArrayAccessor.get_sample_indexer()HaplotypeArrayAccessor.get_variant_indexer()HaplotypeArrayAccessor.to_diploid()HaplotypeArrayAccessor.to_diploid_standardized()
PhenotypeArrayPedigreeArray
- utils module
profiled()ids_from_n_generation()paste()merge_duplicates()ids_from_generation()ids_from_generation_range()exhaustive_permutation()exhaustive_enumerate()sort_and_paste()merge_duplicate_pairs()match()matching_indices_conditional()ensure2D()cartesian_product()VariableCountConstantCountPoissonCountZeroTruncatedPoissonCountNegativeBinomialCountMixtureCountstandardize_array()standardize_array_hw()unique_identifier()cov2cor()to_simplex()to_proportions()print_tree()hierarchical_subsample()