Founder haplotypes

Overview

xftsim simulations can be highly realistic, but they can also be quite simplistic. Founder data is one place we see this: we can use real phased haplotype data or simulate haplotypes as independent Bernoulli trials. As fully synthetic data is extremely convenient to work with, we recommend at the very least using such data to prototype or debug simulations.

In what follows, we first introduce tools for generating haplotypes from scratch, then cover tools for importing external haplotype data.

Haplotypes from scratch

The simplest methods for generating fully synthetic haplotype data involve generating independent Bernoulli trials. Given a set of allele frequencies, founders.founder_haplotypes_from_AFs() will randomly generate a haplotype array using this method:

import xftsim as xft
from xftsim import founders


afs = [0,1,.5]
founders.founder_haplotypes_from_AFs(n=10, afs=afs)

<xarray.DataArray 'HaplotypeArray' (sample: 10, variant: 6)>
array([[0, 0, 1, 1, 1, 1],
       [0, 0, 1, 1, 1, 1],
       [0, 0, 1, 1, 1, 1],
       [0, 0, 1, 1, 0, 1],
       [0, 0, 1, 1, 0, 0],
       [0, 0, 1, 1, 1, 1],
       [0, 0, 1, 1, 1, 0],
       [0, 0, 1, 1, 1, 1],
       [0, 0, 1, 1, 0, 0],
       [0, 0, 1, 1, 1, 0]], dtype=int8)
Coordinates: (12/13)
  * sample       (sample) <U10 '0..0_0.0_0' '0..0_1.0_1' ... '0..0_9.0_9'
    iid          (sample) object '0_0' '0_1' '0_2' '0_3' ... '0_7' '0_8' '0_9'
    fid          (sample) object '0_0' '0_1' '0_2' '0_3' ... '0_7' '0_8' '0_9'
    sex          (sample) int64 0 1 0 1 0 1 0 1 0 1
  * variant      (variant) <U3 '0.0' '0.1' '1.0' '1.1' '2.0' '2.1'
    vid          (variant) object '0' '0' '1' '1' '2' '2'
    ...           ...
    hcopy        (variant) object '0' '1' '0' '1' '0' '1'
    zero_allele  (variant) object 'A' 'A' 'A' 'A' 'A' 'A'
    one_allele   (variant) object 'G' 'G' 'G' 'G' 'G' 'G'
    af           (variant) float64 0.0 0.0 1.0 1.0 0.5 0.5
    pos_bp       (variant) float64 nan nan nan nan nan nan
    pos_cM       (variant) float64 nan nan nan nan nan nan
Attributes:
    generation:  0

xarray.DataArray

'HaplotypeArray'

• sample: 10
• variant: 6

• 0 0 1 1 1 1 0 0 1 1 1 1 0 0 1 1 1 ... 1 1 1 1 0 0 1 1 0 0 0 0 1 1 1 0

  array([[0, 0, 1, 1, 1, 1],
         [0, 0, 1, 1, 1, 1],
         [0, 0, 1, 1, 1, 1],
         [0, 0, 1, 1, 0, 1],
         [0, 0, 1, 1, 0, 0],
         [0, 0, 1, 1, 1, 1],
         [0, 0, 1, 1, 1, 0],
         [0, 0, 1, 1, 1, 1],
         [0, 0, 1, 1, 0, 0],
         [0, 0, 1, 1, 1, 0]], dtype=int8)

• Coordinates: (13)
  • sample
    (sample)
    <U10
    '0..0_0.0_0' ... '0..0_9.0_9'

    array(['0..0_0.0_0', '0..0_1.0_1', '0..0_2.0_2', '0..0_3.0_3', '0..0_4.0_4',
           '0..0_5.0_5', '0..0_6.0_6', '0..0_7.0_7', '0..0_8.0_8', '0..0_9.0_9'],
          dtype='<U10')

  • iid
    (sample)
    object
    '0_0' '0_1' '0_2' ... '0_8' '0_9'

    array(['0_0', '0_1', '0_2', '0_3', '0_4', '0_5', '0_6', '0_7', '0_8',
           '0_9'], dtype=object)

  • fid
    (sample)
    object
    '0_0' '0_1' '0_2' ... '0_8' '0_9'

    array(['0_0', '0_1', '0_2', '0_3', '0_4', '0_5', '0_6', '0_7', '0_8',
           '0_9'], dtype=object)

  • sex
    (sample)
    int64
    0 1 0 1 0 1 0 1 0 1

    array([0, 1, 0, 1, 0, 1, 0, 1, 0, 1])

  • variant
    (variant)
    <U3
    '0.0' '0.1' '1.0' '1.1' '2.0' '2.1'

    array(['0.0', '0.1', '1.0', '1.1', '2.0', '2.1'], dtype='<U3')

  • vid
    (variant)
    object
    '0' '0' '1' '1' '2' '2'

    array(['0', '0', '1', '1', '2', '2'], dtype=object)

  • chrom
    (variant)
    int64
    0 0 1 1 2 2

    array([0, 0, 1, 1, 2, 2])

  • hcopy
    (variant)
    object
    '0' '1' '0' '1' '0' '1'

    array(['0', '1', '0', '1', '0', '1'], dtype=object)

  • zero_allele
    (variant)
    object
    'A' 'A' 'A' 'A' 'A' 'A'

    array(['A', 'A', 'A', 'A', 'A', 'A'], dtype=object)

  • one_allele
    (variant)
    object
    'G' 'G' 'G' 'G' 'G' 'G'

    array(['G', 'G', 'G', 'G', 'G', 'G'], dtype=object)

  • af
    (variant)
    float64
    0.0 0.0 1.0 1.0 0.5 0.5

    array([0. , 0. , 1. , 1. , 0.5, 0.5])

  • pos_bp
    (variant)
    float64
    nan nan nan nan nan nan

    array([nan, nan, nan, nan, nan, nan])

  • pos_cM
    (variant)
    float64
    nan nan nan nan nan nan

    array([nan, nan, nan, nan, nan, nan])

• Indexes: (2)
  • sample
    PandasIndex

    PandasIndex(Index(['0..0_0.0_0', '0..0_1.0_1', '0..0_2.0_2', '0..0_3.0_3', '0..0_4.0_4',
           '0..0_5.0_5', '0..0_6.0_6', '0..0_7.0_7', '0..0_8.0_8', '0..0_9.0_9'],
          dtype='object', name='sample'))

  • variant
    PandasIndex

    PandasIndex(Index(['0.0', '0.1', '1.0', '1.1', '2.0', '2.1'], dtype='object', name='variant'))

• Attributes: (1)

  generation :

  0

For convenience, the function founders.founder_haplotypes_uniform_AFs() will uniformly sample m allele frequencies between minMAF and 1 - minMAF:

founders.founder_haplotypes_uniform_AFs(n=10,m=10,minMAF=.05)

<xarray.DataArray 'HaplotypeArray' (sample: 10, variant: 20)>
array([[1, 0, 0, 1, 1, 1, 0, 0, 0, 0, 1, 0, 0, 0, 1, 1, 0, 0, 1, 0],
       [0, 0, 1, 0, 1, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0],
       [0, 1, 1, 1, 1, 1, 1, 1, 0, 0, 1, 0, 0, 0, 1, 1, 0, 0, 0, 1],
       [0, 1, 1, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 0, 1, 1, 0],
       [0, 1, 1, 0, 0, 1, 1, 1, 0, 0, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0],
       [1, 1, 1, 1, 1, 0, 1, 1, 0, 1, 1, 1, 0, 1, 0, 1, 1, 0, 0, 0],
       [1, 1, 0, 1, 0, 1, 1, 0, 0, 0, 0, 1, 1, 0, 1, 1, 0, 0, 0, 0],
       [1, 1, 1, 1, 1, 1, 0, 1, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 1, 0],
       [1, 1, 0, 1, 1, 1, 1, 0, 0, 1, 1, 0, 1, 0, 1, 1, 0, 0, 0, 0],
       [0, 0, 1, 0, 1, 1, 1, 1, 0, 0, 1, 1, 0, 1, 1, 1, 0, 1, 1, 0]],
      dtype=int8)
Coordinates: (12/13)
  * sample       (sample) <U10 '0..0_0.0_0' '0..0_1.0_1' ... '0..0_9.0_9'
    iid          (sample) object '0_0' '0_1' '0_2' '0_3' ... '0_7' '0_8' '0_9'
    fid          (sample) object '0_0' '0_1' '0_2' '0_3' ... '0_7' '0_8' '0_9'
    sex          (sample) int64 0 1 0 1 0 1 0 1 0 1
  * variant      (variant) <U3 '0.0' '0.1' '1.0' '1.1' ... '8.1' '9.0' '9.1'
    vid          (variant) object '0' '0' '1' '1' '2' ... '7' '8' '8' '9' '9'
    ...           ...
    hcopy        (variant) object '0' '1' '0' '1' '0' ... '1' '0' '1' '0' '1'
    zero_allele  (variant) object 'A' 'A' 'A' 'A' 'A' ... 'A' 'A' 'A' 'A' 'A'
    one_allele   (variant) object 'G' 'G' 'G' 'G' 'G' ... 'G' 'G' 'G' 'G' 'G'
    af           (variant) float64 0.5635 0.5635 0.7249 ... 0.1328 0.1955 0.1955
    pos_bp       (variant) float64 nan nan nan nan nan ... nan nan nan nan nan
    pos_cM       (variant) float64 nan nan nan nan nan ... nan nan nan nan nan
Attributes:
    generation:  0

xarray.DataArray

'HaplotypeArray'

• sample: 10
• variant: 20

• 1 0 0 1 1 1 0 0 0 0 1 0 0 0 1 1 0 ... 1 1 1 1 0 0 1 1 0 1 1 1 0 1 1 0

  array([[1, 0, 0, 1, 1, 1, 0, 0, 0, 0, 1, 0, 0, 0, 1, 1, 0, 0, 1, 0],
         [0, 0, 1, 0, 1, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0],
         [0, 1, 1, 1, 1, 1, 1, 1, 0, 0, 1, 0, 0, 0, 1, 1, 0, 0, 0, 1],
         [0, 1, 1, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 0, 1, 1, 0],
         [0, 1, 1, 0, 0, 1, 1, 1, 0, 0, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0],
         [1, 1, 1, 1, 1, 0, 1, 1, 0, 1, 1, 1, 0, 1, 0, 1, 1, 0, 0, 0],
         [1, 1, 0, 1, 0, 1, 1, 0, 0, 0, 0, 1, 1, 0, 1, 1, 0, 0, 0, 0],
         [1, 1, 1, 1, 1, 1, 0, 1, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 1, 0],
         [1, 1, 0, 1, 1, 1, 1, 0, 0, 1, 1, 0, 1, 0, 1, 1, 0, 0, 0, 0],
         [0, 0, 1, 0, 1, 1, 1, 1, 0, 0, 1, 1, 0, 1, 1, 1, 0, 1, 1, 0]],
        dtype=int8)

• Coordinates: (13)
  • sample
    (sample)
    <U10
    '0..0_0.0_0' ... '0..0_9.0_9'

    array(['0..0_0.0_0', '0..0_1.0_1', '0..0_2.0_2', '0..0_3.0_3', '0..0_4.0_4',
           '0..0_5.0_5', '0..0_6.0_6', '0..0_7.0_7', '0..0_8.0_8', '0..0_9.0_9'],
          dtype='<U10')

  • iid
    (sample)
    object
    '0_0' '0_1' '0_2' ... '0_8' '0_9'

    array(['0_0', '0_1', '0_2', '0_3', '0_4', '0_5', '0_6', '0_7', '0_8',
           '0_9'], dtype=object)

  • fid
    (sample)
    object
    '0_0' '0_1' '0_2' ... '0_8' '0_9'

    array(['0_0', '0_1', '0_2', '0_3', '0_4', '0_5', '0_6', '0_7', '0_8',
           '0_9'], dtype=object)

  • sex
    (sample)
    int64
    0 1 0 1 0 1 0 1 0 1

    array([0, 1, 0, 1, 0, 1, 0, 1, 0, 1])

  • variant
    (variant)
    <U3
    '0.0' '0.1' '1.0' ... '9.0' '9.1'

    array(['0.0', '0.1', '1.0', '1.1', '2.0', '2.1', '3.0', '3.1', '4.0', '4.1',
           '5.0', '5.1', '6.0', '6.1', '7.0', '7.1', '8.0', '8.1', '9.0', '9.1'],
          dtype='<U3')

  • vid
    (variant)
    object
    '0' '0' '1' '1' ... '8' '8' '9' '9'

    array(['0', '0', '1', '1', '2', '2', '3', '3', '4', '4', '5', '5', '6',
           '6', '7', '7', '8', '8', '9', '9'], dtype=object)

  • chrom
    (variant)
    int64
    0 0 1 1 2 2 3 3 ... 6 6 7 7 8 8 9 9

    array([0, 0, 1, 1, 2, 2, 3, 3, 4, 4, 5, 5, 6, 6, 7, 7, 8, 8, 9, 9])

  • hcopy
    (variant)
    object
    '0' '1' '0' '1' ... '0' '1' '0' '1'

    array(['0', '1', '0', '1', '0', '1', '0', '1', '0', '1', '0', '1', '0',
           '1', '0', '1', '0', '1', '0', '1'], dtype=object)

  • zero_allele
    (variant)
    object
    'A' 'A' 'A' 'A' ... 'A' 'A' 'A' 'A'

    array(['A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A',
           'A', 'A', 'A', 'A', 'A', 'A', 'A'], dtype=object)

  • one_allele
    (variant)
    object
    'G' 'G' 'G' 'G' ... 'G' 'G' 'G' 'G'

    array(['G', 'G', 'G', 'G', 'G', 'G', 'G', 'G', 'G', 'G', 'G', 'G', 'G',
           'G', 'G', 'G', 'G', 'G', 'G', 'G'], dtype=object)

  • af
    (variant)
    float64
    0.5635 0.5635 ... 0.1955 0.1955

    array([0.56350357, 0.56350357, 0.72489337, 0.72489337, 0.72599522,
           0.72599522, 0.75131873, 0.75131873, 0.13939728, 0.13939728,
           0.51194756, 0.51194756, 0.4945147 , 0.4945147 , 0.73179852,
           0.73179852, 0.13280441, 0.13280441, 0.19545121, 0.19545121])

  • pos_bp
    (variant)
    float64
    nan nan nan nan ... nan nan nan nan

    array([nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
           nan, nan, nan, nan, nan, nan, nan])

  • pos_cM
    (variant)
    float64
    nan nan nan nan ... nan nan nan nan

    array([nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
           nan, nan, nan, nan, nan, nan, nan])

• Indexes: (2)
  • sample
    PandasIndex

    PandasIndex(Index(['0..0_0.0_0', '0..0_1.0_1', '0..0_2.0_2', '0..0_3.0_3', '0..0_4.0_4',
           '0..0_5.0_5', '0..0_6.0_6', '0..0_7.0_7', '0..0_8.0_8', '0..0_9.0_9'],
          dtype='object', name='sample'))

  • variant
    PandasIndex

    PandasIndex(Index(['0.0', '0.1', '1.0', '1.1', '2.0', '2.1', '3.0', '3.1', '4.0', '4.1',
           '5.0', '5.1', '6.0', '6.1', '7.0', '7.1', '8.0', '8.1', '9.0', '9.1'],
          dtype='object', name='variant'))

• Attributes: (1)

  generation :

  0

When creating a generic xft.Index.VariantIndex (to assign variant IDs etc) as in the above examples, xftsim will attempt to evenly divide variants between (up to) 22 chromosomes.

Of course, we can always directly provide haplotypes to the xft.struct.HaplotypeArray constructor:

import numpy as np

arbitrary_haplotypes = np.random.binomial(1,.3, (100,200))
xft.struct.HaplotypeArray(arbitrary_haplotypes)

/home/rsb/Dropbox/ftsim/xftsim/xftsim/struct.py:1280: UserWarning: Using empirical allele frequencies as variant_indexer not provided
  warnings.warn(

<xarray.DataArray 'HaplotypeArray' (sample: 100, variant: 200)>
array([[0, 0, 0, ..., 0, 1, 1],
       [1, 1, 0, ..., 0, 0, 0],
       [1, 0, 0, ..., 0, 0, 0],
       ...,
       [1, 0, 0, ..., 0, 0, 0],
       [0, 0, 0, ..., 0, 0, 1],
       [0, 0, 0, ..., 0, 0, 1]], dtype=int8)
Coordinates: (12/13)
  * sample       (sample) <U12 '0..0_0.0_0' '0..0_1.0_1' ... '0..0_99.0_99'
    iid          (sample) object '0_0' '0_1' '0_2' ... '0_97' '0_98' '0_99'
    fid          (sample) object '0_0' '0_1' '0_2' ... '0_97' '0_98' '0_99'
    sex          (sample) int64 0 1 0 1 0 1 0 1 0 1 0 ... 1 0 1 0 1 0 1 0 1 0 1
  * variant      (variant) <U4 '0.0' '0.1' '1.0' '1.1' ... '98.1' '99.0' '99.1'
    vid          (variant) object '0' '0' '1' '1' '2' ... '98' '98' '99' '99'
    ...           ...
    hcopy        (variant) object '0' '1' '0' '1' '0' ... '1' '0' '1' '0' '1'
    zero_allele  (variant) object 'A' 'A' 'A' 'A' 'A' ... 'A' 'A' 'A' 'A' 'A'
    one_allele   (variant) object 'G' 'G' 'G' 'G' 'G' ... 'G' 'G' 'G' 'G' 'G'
    af           (variant) float64 0.315 0.315 0.265 0.265 ... 0.295 0.34 0.34
    pos_bp       (variant) float64 nan nan nan nan nan ... nan nan nan nan nan
    pos_cM       (variant) float64 nan nan nan nan nan ... nan nan nan nan nan
Attributes:
    generation:  0

xarray.DataArray

'HaplotypeArray'

• sample: 100
• variant: 200

• 0 0 0 0 1 0 0 0 1 0 0 0 0 0 0 1 0 ... 0 0 1 0 0 1 0 0 0 0 0 0 0 0 0 1

  array([[0, 0, 0, ..., 0, 1, 1],
         [1, 1, 0, ..., 0, 0, 0],
         [1, 0, 0, ..., 0, 0, 0],
         ...,
         [1, 0, 0, ..., 0, 0, 0],
         [0, 0, 0, ..., 0, 0, 1],
         [0, 0, 0, ..., 0, 0, 1]], dtype=int8)

• Coordinates: (13)
  • sample
    (sample)
    <U12
    '0..0_0.0_0' ... '0..0_99.0_99'

    array(['0..0_0.0_0', '0..0_1.0_1', '0..0_2.0_2', '0..0_3.0_3', '0..0_4.0_4',
           '0..0_5.0_5', '0..0_6.0_6', '0..0_7.0_7', '0..0_8.0_8', '0..0_9.0_9',
           '0..0_10.0_10', '0..0_11.0_11', '0..0_12.0_12', '0..0_13.0_13',
           '0..0_14.0_14', '0..0_15.0_15', '0..0_16.0_16', '0..0_17.0_17',
           '0..0_18.0_18', '0..0_19.0_19', '0..0_20.0_20', '0..0_21.0_21',
           '0..0_22.0_22', '0..0_23.0_23', '0..0_24.0_24', '0..0_25.0_25',
           '0..0_26.0_26', '0..0_27.0_27', '0..0_28.0_28', '0..0_29.0_29',
           '0..0_30.0_30', '0..0_31.0_31', '0..0_32.0_32', '0..0_33.0_33',
           '0..0_34.0_34', '0..0_35.0_35', '0..0_36.0_36', '0..0_37.0_37',
           '0..0_38.0_38', '0..0_39.0_39', '0..0_40.0_40', '0..0_41.0_41',
           '0..0_42.0_42', '0..0_43.0_43', '0..0_44.0_44', '0..0_45.0_45',
           '0..0_46.0_46', '0..0_47.0_47', '0..0_48.0_48', '0..0_49.0_49',
           '0..0_50.0_50', '0..0_51.0_51', '0..0_52.0_52', '0..0_53.0_53',
           '0..0_54.0_54', '0..0_55.0_55', '0..0_56.0_56', '0..0_57.0_57',
           '0..0_58.0_58', '0..0_59.0_59', '0..0_60.0_60', '0..0_61.0_61',
           '0..0_62.0_62', '0..0_63.0_63', '0..0_64.0_64', '0..0_65.0_65',
           '0..0_66.0_66', '0..0_67.0_67', '0..0_68.0_68', '0..0_69.0_69',
           '0..0_70.0_70', '0..0_71.0_71', '0..0_72.0_72', '0..0_73.0_73',
           '0..0_74.0_74', '0..0_75.0_75', '0..0_76.0_76', '0..0_77.0_77',
           '0..0_78.0_78', '0..0_79.0_79', '0..0_80.0_80', '0..0_81.0_81',
           '0..0_82.0_82', '0..0_83.0_83', '0..0_84.0_84', '0..0_85.0_85',
           '0..0_86.0_86', '0..0_87.0_87', '0..0_88.0_88', '0..0_89.0_89',
           '0..0_90.0_90', '0..0_91.0_91', '0..0_92.0_92', '0..0_93.0_93',
           '0..0_94.0_94', '0..0_95.0_95', '0..0_96.0_96', '0..0_97.0_97',
           '0..0_98.0_98', '0..0_99.0_99'], dtype='<U12')

  • iid
    (sample)
    object
    '0_0' '0_1' '0_2' ... '0_98' '0_99'

    array(['0_0', '0_1', '0_2', '0_3', '0_4', '0_5', '0_6', '0_7', '0_8',
           '0_9', '0_10', '0_11', '0_12', '0_13', '0_14', '0_15', '0_16',
           '0_17', '0_18', '0_19', '0_20', '0_21', '0_22', '0_23', '0_24',
           '0_25', '0_26', '0_27', '0_28', '0_29', '0_30', '0_31', '0_32',
           '0_33', '0_34', '0_35', '0_36', '0_37', '0_38', '0_39', '0_40',
           '0_41', '0_42', '0_43', '0_44', '0_45', '0_46', '0_47', '0_48',
           '0_49', '0_50', '0_51', '0_52', '0_53', '0_54', '0_55', '0_56',
           '0_57', '0_58', '0_59', '0_60', '0_61', '0_62', '0_63', '0_64',
           '0_65', '0_66', '0_67', '0_68', '0_69', '0_70', '0_71', '0_72',
           '0_73', '0_74', '0_75', '0_76', '0_77', '0_78', '0_79', '0_80',
           '0_81', '0_82', '0_83', '0_84', '0_85', '0_86', '0_87', '0_88',
           '0_89', '0_90', '0_91', '0_92', '0_93', '0_94', '0_95', '0_96',
           '0_97', '0_98', '0_99'], dtype=object)

  • fid
    (sample)
    object
    '0_0' '0_1' '0_2' ... '0_98' '0_99'

    array(['0_0', '0_1', '0_2', '0_3', '0_4', '0_5', '0_6', '0_7', '0_8',
           '0_9', '0_10', '0_11', '0_12', '0_13', '0_14', '0_15', '0_16',
           '0_17', '0_18', '0_19', '0_20', '0_21', '0_22', '0_23', '0_24',
           '0_25', '0_26', '0_27', '0_28', '0_29', '0_30', '0_31', '0_32',
           '0_33', '0_34', '0_35', '0_36', '0_37', '0_38', '0_39', '0_40',
           '0_41', '0_42', '0_43', '0_44', '0_45', '0_46', '0_47', '0_48',
           '0_49', '0_50', '0_51', '0_52', '0_53', '0_54', '0_55', '0_56',
           '0_57', '0_58', '0_59', '0_60', '0_61', '0_62', '0_63', '0_64',
           '0_65', '0_66', '0_67', '0_68', '0_69', '0_70', '0_71', '0_72',
           '0_73', '0_74', '0_75', '0_76', '0_77', '0_78', '0_79', '0_80',
           '0_81', '0_82', '0_83', '0_84', '0_85', '0_86', '0_87', '0_88',
           '0_89', '0_90', '0_91', '0_92', '0_93', '0_94', '0_95', '0_96',
           '0_97', '0_98', '0_99'], dtype=object)

  • sex
    (sample)
    int64
    0 1 0 1 0 1 0 1 ... 0 1 0 1 0 1 0 1

    array([0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1,
           0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1,
           0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1,
           0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1,
           0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1])

  • variant
    (variant)
    <U4
    '0.0' '0.1' '1.0' ... '99.0' '99.1'

    array(['0.0', '0.1', '1.0', '1.1', '2.0', '2.1', '3.0', '3.1', '4.0', '4.1',
           '5.0', '5.1', '6.0', '6.1', '7.0', '7.1', '8.0', '8.1', '9.0', '9.1',
           '10.0', '10.1', '11.0', '11.1', '12.0', '12.1', '13.0', '13.1', '14.0',
           '14.1', '15.0', '15.1', '16.0', '16.1', '17.0', '17.1', '18.0', '18.1',
           '19.0', '19.1', '20.0', '20.1', '21.0', '21.1', '22.0', '22.1', '23.0',
           '23.1', '24.0', '24.1', '25.0', '25.1', '26.0', '26.1', '27.0', '27.1',
           '28.0', '28.1', '29.0', '29.1', '30.0', '30.1', '31.0', '31.1', '32.0',
           '32.1', '33.0', '33.1', '34.0', '34.1', '35.0', '35.1', '36.0', '36.1',
           '37.0', '37.1', '38.0', '38.1', '39.0', '39.1', '40.0', '40.1', '41.0',
           '41.1', '42.0', '42.1', '43.0', '43.1', '44.0', '44.1', '45.0', '45.1',
           '46.0', '46.1', '47.0', '47.1', '48.0', '48.1', '49.0', '49.1', '50.0',
           '50.1', '51.0', '51.1', '52.0', '52.1', '53.0', '53.1', '54.0', '54.1',
           '55.0', '55.1', '56.0', '56.1', '57.0', '57.1', '58.0', '58.1', '59.0',
           '59.1', '60.0', '60.1', '61.0', '61.1', '62.0', '62.1', '63.0', '63.1',
           '64.0', '64.1', '65.0', '65.1', '66.0', '66.1', '67.0', '67.1', '68.0',
           '68.1', '69.0', '69.1', '70.0', '70.1', '71.0', '71.1', '72.0', '72.1',
           '73.0', '73.1', '74.0', '74.1', '75.0', '75.1', '76.0', '76.1', '77.0',
           '77.1', '78.0', '78.1', '79.0', '79.1', '80.0', '80.1', '81.0', '81.1',
           '82.0', '82.1', '83.0', '83.1', '84.0', '84.1', '85.0', '85.1', '86.0',
           '86.1', '87.0', '87.1', '88.0', '88.1', '89.0', '89.1', '90.0', '90.1',
           '91.0', '91.1', '92.0', '92.1', '93.0', '93.1', '94.0', '94.1', '95.0',
           '95.1', '96.0', '96.1', '97.0', '97.1', '98.0', '98.1', '99.0', '99.1'],
          dtype='<U4')

  • vid
    (variant)
    object
    '0' '0' '1' '1' ... '98' '99' '99'

    array(['0', '0', '1', '1', '2', '2', '3', '3', '4', '4', '5', '5', '6',
           '6', '7', '7', '8', '8', '9', '9', '10', '10', '11', '11', '12',
           '12', '13', '13', '14', '14', '15', '15', '16', '16', '17', '17',
           '18', '18', '19', '19', '20', '20', '21', '21', '22', '22', '23',
           '23', '24', '24', '25', '25', '26', '26', '27', '27', '28', '28',
           '29', '29', '30', '30', '31', '31', '32', '32', '33', '33', '34',
           '34', '35', '35', '36', '36', '37', '37', '38', '38', '39', '39',
           '40', '40', '41', '41', '42', '42', '43', '43', '44', '44', '45',
           '45', '46', '46', '47', '47', '48', '48', '49', '49', '50', '50',
           '51', '51', '52', '52', '53', '53', '54', '54', '55', '55', '56',
           '56', '57', '57', '58', '58', '59', '59', '60', '60', '61', '61',
           '62', '62', '63', '63', '64', '64', '65', '65', '66', '66', '67',
           '67', '68', '68', '69', '69', '70', '70', '71', '71', '72', '72',
           '73', '73', '74', '74', '75', '75', '76', '76', '77', '77', '78',
           '78', '79', '79', '80', '80', '81', '81', '82', '82', '83', '83',
           '84', '84', '85', '85', '86', '86', '87', '87', '88', '88', '89',
           '89', '90', '90', '91', '91', '92', '92', '93', '93', '94', '94',
           '95', '95', '96', '96', '97', '97', '98', '98', '99', '99'],
          dtype=object)

  • chrom
    (variant)
    int64
    0 0 0 0 0 0 0 ... 19 19 19 19 19 19

    array([ 0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  1,  1,  1,  1,  1,  1,  1,
            1,  1,  1,  2,  2,  2,  2,  2,  2,  2,  2,  2,  2,  3,  3,  3,  3,
            3,  3,  3,  3,  3,  3,  4,  4,  4,  4,  4,  4,  4,  4,  4,  4,  5,
            5,  5,  5,  5,  5,  5,  5,  5,  5,  6,  6,  6,  6,  6,  6,  6,  6,
            6,  6,  7,  7,  7,  7,  7,  7,  7,  7,  7,  7,  8,  8,  8,  8,  8,
            8,  8,  8,  8,  8,  9,  9,  9,  9,  9,  9,  9,  9,  9,  9, 10, 10,
           10, 10, 10, 10, 10, 10, 10, 10, 11, 11, 11, 11, 11, 11, 11, 11, 11,
           11, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 13, 13, 13, 13, 13, 13,
           13, 13, 13, 13, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 15, 15, 15,
           15, 15, 15, 15, 15, 15, 15, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16,
           17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 18, 18, 18, 18, 18, 18, 18,
           18, 18, 18, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19])

  • hcopy
    (variant)
    object
    '0' '1' '0' '1' ... '0' '1' '0' '1'

    array(['0', '1', '0', '1', '0', '1', '0', '1', '0', '1', '0', '1', '0',
           '1', '0', '1', '0', '1', '0', '1', '0', '1', '0', '1', '0', '1',
           '0', '1', '0', '1', '0', '1', '0', '1', '0', '1', '0', '1', '0',
           '1', '0', '1', '0', '1', '0', '1', '0', '1', '0', '1', '0', '1',
           '0', '1', '0', '1', '0', '1', '0', '1', '0', '1', '0', '1', '0',
           '1', '0', '1', '0', '1', '0', '1', '0', '1', '0', '1', '0', '1',
           '0', '1', '0', '1', '0', '1', '0', '1', '0', '1', '0', '1', '0',
           '1', '0', '1', '0', '1', '0', '1', '0', '1', '0', '1', '0', '1',
           '0', '1', '0', '1', '0', '1', '0', '1', '0', '1', '0', '1', '0',
           '1', '0', '1', '0', '1', '0', '1', '0', '1', '0', '1', '0', '1',
           '0', '1', '0', '1', '0', '1', '0', '1', '0', '1', '0', '1', '0',
           '1', '0', '1', '0', '1', '0', '1', '0', '1', '0', '1', '0', '1',
           '0', '1', '0', '1', '0', '1', '0', '1', '0', '1', '0', '1', '0',
           '1', '0', '1', '0', '1', '0', '1', '0', '1', '0', '1', '0', '1',
           '0', '1', '0', '1', '0', '1', '0', '1', '0', '1', '0', '1', '0',
           '1', '0', '1', '0', '1'], dtype=object)

  • zero_allele
    (variant)
    object
    'A' 'A' 'A' 'A' ... 'A' 'A' 'A' 'A'

    array(['A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A',
           'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A',
           'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A',
           'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A',
           'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A',
           'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A',
           'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A',
           'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A',
           'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A',
           'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A',
           'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A',
           'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A',
           'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A',
           'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A',
           'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A',
           'A', 'A', 'A', 'A', 'A'], dtype=object)

  • one_allele
    (variant)
    object
    'G' 'G' 'G' 'G' ... 'G' 'G' 'G' 'G'

    array(['G', 'G', 'G', 'G', 'G', 'G', 'G', 'G', 'G', 'G', 'G', 'G', 'G',
           'G', 'G', 'G', 'G', 'G', 'G', 'G', 'G', 'G', 'G', 'G', 'G', 'G',
           'G', 'G', 'G', 'G', 'G', 'G', 'G', 'G', 'G', 'G', 'G', 'G', 'G',
           'G', 'G', 'G', 'G', 'G', 'G', 'G', 'G', 'G', 'G', 'G', 'G', 'G',
           'G', 'G', 'G', 'G', 'G', 'G', 'G', 'G', 'G', 'G', 'G', 'G', 'G',
           'G', 'G', 'G', 'G', 'G', 'G', 'G', 'G', 'G', 'G', 'G', 'G', 'G',
           'G', 'G', 'G', 'G', 'G', 'G', 'G', 'G', 'G', 'G', 'G', 'G', 'G',
           'G', 'G', 'G', 'G', 'G', 'G', 'G', 'G', 'G', 'G', 'G', 'G', 'G',
           'G', 'G', 'G', 'G', 'G', 'G', 'G', 'G', 'G', 'G', 'G', 'G', 'G',
           'G', 'G', 'G', 'G', 'G', 'G', 'G', 'G', 'G', 'G', 'G', 'G', 'G',
           'G', 'G', 'G', 'G', 'G', 'G', 'G', 'G', 'G', 'G', 'G', 'G', 'G',
           'G', 'G', 'G', 'G', 'G', 'G', 'G', 'G', 'G', 'G', 'G', 'G', 'G',
           'G', 'G', 'G', 'G', 'G', 'G', 'G', 'G', 'G', 'G', 'G', 'G', 'G',
           'G', 'G', 'G', 'G', 'G', 'G', 'G', 'G', 'G', 'G', 'G', 'G', 'G',
           'G', 'G', 'G', 'G', 'G', 'G', 'G', 'G', 'G', 'G', 'G', 'G', 'G',
           'G', 'G', 'G', 'G', 'G'], dtype=object)

  • af
    (variant)
    float64
    0.315 0.315 0.265 ... 0.34 0.34

    array([0.315, 0.315, 0.265, 0.265, 0.285, 0.285, 0.31 , 0.31 , 0.335,
           0.335, 0.29 , 0.29 , 0.32 , 0.32 , 0.365, 0.365, 0.24 , 0.24 ,
           0.335, 0.335, 0.23 , 0.23 , 0.315, 0.315, 0.305, 0.305, 0.35 ,
           0.35 , 0.3  , 0.3  , 0.265, 0.265, 0.275, 0.275, 0.275, 0.275,
           0.26 , 0.26 , 0.31 , 0.31 , 0.305, 0.305, 0.3  , 0.3  , 0.33 ,
           0.33 , 0.285, 0.285, 0.31 , 0.31 , 0.375, 0.375, 0.305, 0.305,
           0.295, 0.295, 0.325, 0.325, 0.315, 0.315, 0.27 , 0.27 , 0.24 ,
           0.24 , 0.32 , 0.32 , 0.27 , 0.27 , 0.315, 0.315, 0.31 , 0.31 ,
           0.34 , 0.34 , 0.295, 0.295, 0.265, 0.265, 0.3  , 0.3  , 0.305,
           0.305, 0.265, 0.265, 0.335, 0.335, 0.355, 0.355, 0.345, 0.345,
           0.31 , 0.31 , 0.235, 0.235, 0.365, 0.365, 0.285, 0.285, 0.29 ,
           0.29 , 0.25 , 0.25 , 0.265, 0.265, 0.29 , 0.29 , 0.365, 0.365,
           0.315, 0.315, 0.285, 0.285, 0.355, 0.355, 0.235, 0.235, 0.325,
           0.325, 0.33 , 0.33 , 0.31 , 0.31 , 0.355, 0.355, 0.33 , 0.33 ,
           0.305, 0.305, 0.32 , 0.32 , 0.3  , 0.3  , 0.32 , 0.32 , 0.315,
           0.315, 0.26 , 0.26 , 0.29 , 0.29 , 0.285, 0.285, 0.315, 0.315,
           0.31 , 0.31 , 0.325, 0.325, 0.35 , 0.35 , 0.32 , 0.32 , 0.34 ,
           0.34 , 0.28 , 0.28 , 0.265, 0.265, 0.35 , 0.35 , 0.285, 0.285,
           0.335, 0.335, 0.255, 0.255, 0.325, 0.325, 0.325, 0.325, 0.28 ,
           0.28 , 0.245, 0.245, 0.325, 0.325, 0.34 , 0.34 , 0.275, 0.275,
           0.345, 0.345, 0.335, 0.335, 0.31 , 0.31 , 0.24 , 0.24 , 0.275,
           0.275, 0.33 , 0.33 , 0.32 , 0.32 , 0.32 , 0.32 , 0.295, 0.295,
           0.34 , 0.34 ])

  • pos_bp
    (variant)
    float64
    nan nan nan nan ... nan nan nan nan

    array([nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
           nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
           nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
           nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
           nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
           nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
           nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
           nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
           nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
           nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
           nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
           nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
           nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
           nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
           nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
           nan, nan, nan, nan, nan])

  • pos_cM
    (variant)
    float64
    nan nan nan nan ... nan nan nan nan

    array([nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
           nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
           nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
           nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
           nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
           nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
           nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
           nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
           nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
           nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
           nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
           nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
           nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
           nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
           nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
           nan, nan, nan, nan, nan])

• Indexes: (2)
  • sample
    PandasIndex

    PandasIndex(Index(['0..0_0.0_0', '0..0_1.0_1', '0..0_2.0_2', '0..0_3.0_3', '0..0_4.0_4',
           '0..0_5.0_5', '0..0_6.0_6', '0..0_7.0_7', '0..0_8.0_8', '0..0_9.0_9',
           '0..0_10.0_10', '0..0_11.0_11', '0..0_12.0_12', '0..0_13.0_13',
           '0..0_14.0_14', '0..0_15.0_15', '0..0_16.0_16', '0..0_17.0_17',
           '0..0_18.0_18', '0..0_19.0_19', '0..0_20.0_20', '0..0_21.0_21',
           '0..0_22.0_22', '0..0_23.0_23', '0..0_24.0_24', '0..0_25.0_25',
           '0..0_26.0_26', '0..0_27.0_27', '0..0_28.0_28', '0..0_29.0_29',
           '0..0_30.0_30', '0..0_31.0_31', '0..0_32.0_32', '0..0_33.0_33',
           '0..0_34.0_34', '0..0_35.0_35', '0..0_36.0_36', '0..0_37.0_37',
           '0..0_38.0_38', '0..0_39.0_39', '0..0_40.0_40', '0..0_41.0_41',
           '0..0_42.0_42', '0..0_43.0_43', '0..0_44.0_44', '0..0_45.0_45',
           '0..0_46.0_46', '0..0_47.0_47', '0..0_48.0_48', '0..0_49.0_49',
           '0..0_50.0_50', '0..0_51.0_51', '0..0_52.0_52', '0..0_53.0_53',
           '0..0_54.0_54', '0..0_55.0_55', '0..0_56.0_56', '0..0_57.0_57',
           '0..0_58.0_58', '0..0_59.0_59', '0..0_60.0_60', '0..0_61.0_61',
           '0..0_62.0_62', '0..0_63.0_63', '0..0_64.0_64', '0..0_65.0_65',
           '0..0_66.0_66', '0..0_67.0_67', '0..0_68.0_68', '0..0_69.0_69',
           '0..0_70.0_70', '0..0_71.0_71', '0..0_72.0_72', '0..0_73.0_73',
           '0..0_74.0_74', '0..0_75.0_75', '0..0_76.0_76', '0..0_77.0_77',
           '0..0_78.0_78', '0..0_79.0_79', '0..0_80.0_80', '0..0_81.0_81',
           '0..0_82.0_82', '0..0_83.0_83', '0..0_84.0_84', '0..0_85.0_85',
           '0..0_86.0_86', '0..0_87.0_87', '0..0_88.0_88', '0..0_89.0_89',
           '0..0_90.0_90', '0..0_91.0_91', '0..0_92.0_92', '0..0_93.0_93',
           '0..0_94.0_94', '0..0_95.0_95', '0..0_96.0_96', '0..0_97.0_97',
           '0..0_98.0_98', '0..0_99.0_99'],
          dtype='object', name='sample'))

  • variant
    PandasIndex

    PandasIndex(Index(['0.0', '0.1', '1.0', '1.1', '2.0', '2.1', '3.0', '3.1', '4.0', '4.1',
           ...
           '95.0', '95.1', '96.0', '96.1', '97.0', '97.1', '98.0', '98.1', '99.0',
           '99.1'],
          dtype='object', name='variant', length=200))

• Attributes: (1)

  generation :

  0

Haplotypes from external datasets

We currently support the plink binary (bfile) and variant call (VCF) formats by interfacing with the pandas-plink and sgkit libraries, respectively. We hope to add support for bgen and plink2 (pfile) formats in the future.

plink bfiles

The plink bfile format is inherantly diploid and will break phasing. Thus, when reading in plink data, haplotypes at heterozygous loci are assignmened randomly.

For this example, we’ll use the example data that’s packaged with the pandas-plink library:

from pandas_plink import get_data_folder
from os.path import join

pdat = founders.founder_haplotypes_from_plink_bfile(join(get_data_folder(), "chr*.bed"))
pdat

/home/rsb/miniconda3/lib/python3.9/site-packages/pandas_plink/_read.py:288: UserWarning: More than one FAM file has been specified. Only the first one will be considered.
  warnings.warn(msg, UserWarning)
Mapping files: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 5/5 [00:00<00:00, 344.31it/s]
Mapping files:  83%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▏                          | 5/6 [00:00<00:00, 345.12it/s]

Multiple files read in this order: ['chr11', 'chr12']

<xarray.DataArray 'HaplotypeArray' (sample: 14, variant: 2504)>
dask.array<_genotypes_to_pseudo_haplotypes, shape=(14, 2504), dtype=int8, chunksize=(14, 1558), chunktype=numpy.ndarray>
Coordinates: (12/13)
  * sample       (sample) <U12 '0..B001.B001' '0..B002.B002' ... '0..B014.B014'
    iid          (sample) object 'B001' 'B002' 'B003' ... 'B012' 'B013' 'B014'
    fid          (sample) object 'B001' 'B002' 'B003' ... 'B012' 'B013' 'B014'
    sex          (sample) int64 2 2 2 2 2 2 2 2 2 2 2 2 2 2
  * variant      (variant) <U11 '316849996.0' '316849996.1' ... '373081507.1'
    vid          (variant) object '316849996' '316849996' ... '373081507'
    ...           ...
    hcopy        (variant) object '0' '1' '0' '1' '0' ... '1' '0' '1' '0' '1'
    zero_allele  (variant) object 'C' 'C' 'G' 'G' 'G' ... 'A' 'T' 'T' 'G' 'G'
    one_allele   (variant) object 'T' 'T' 'C' 'C' 'C' ... 'T' 'C' 'C' 'A' 'A'
    af           (variant) float64 nan nan nan nan nan ... nan nan nan nan nan
    pos_bp       (variant) int32 157439 157439 181802 ... 27367844 27367844
    pos_cM       (variant) float64 0.0 0.0 0.0 0.0 0.0 ... 0.0 0.0 0.0 0.0 0.0
Attributes:
    generation:  0

xarray.DataArray

'HaplotypeArray'

• sample: 14
• variant: 2504

• dask.array<chunksize=(14, 1558), meta=np.ndarray>

  Array Chunk Bytes 34.23 kiB 21.30 kiB Shape (14, 2504) (14, 1558) Dask graph 2 chunks in 11 graph layers Data type int8 numpy.ndarray

• Coordinates: (13)
  • sample
    (sample)
    <U12
    '0..B001.B001' ... '0..B014.B014'

    array(['0..B001.B001', '0..B002.B002', '0..B003.B003', '0..B004.B004',
           '0..B005.B005', '0..B006.B006', '0..B007.B007', '0..B008.B008',
           '0..B009.B009', '0..B010.B010', '0..B011.B011', '0..B012.B012',
           '0..B013.B013', '0..B014.B014'], dtype='<U12')

  • iid
    (sample)
    object
    'B001' 'B002' ... 'B013' 'B014'

    array(['B001', 'B002', 'B003', 'B004', 'B005', 'B006', 'B007', 'B008',
           'B009', 'B010', 'B011', 'B012', 'B013', 'B014'], dtype=object)

  • fid
    (sample)
    object
    'B001' 'B002' ... 'B013' 'B014'

    array(['B001', 'B002', 'B003', 'B004', 'B005', 'B006', 'B007', 'B008',
           'B009', 'B010', 'B011', 'B012', 'B013', 'B014'], dtype=object)

  • sex
    (sample)
    int64
    2 2 2 2 2 2 2 2 2 2 2 2 2 2

    array([2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2])

  • variant
    (variant)
    <U11
    '316849996.0' ... '373081507.1'

    array(['316849996.0', '316849996.1', '316874359.0', ..., '372918788.1',
           '373081507.0', '373081507.1'], dtype='<U11')

  • vid
    (variant)
    object
    '316849996' ... '373081507'

    array(['316849996', '316849996', '316874359', ..., '372918788',
           '373081507', '373081507'], dtype=object)

  • chrom
    (variant)
    object
    '11' '11' '11' ... '12' '12' '12'

    array(['11', '11', '11', ..., '12', '12', '12'], dtype=object)

  • hcopy
    (variant)
    object
    '0' '1' '0' '1' ... '0' '1' '0' '1'

    array(['0', '1', '0', ..., '1', '0', '1'], dtype=object)

  • zero_allele
    (variant)
    object
    'C' 'C' 'G' 'G' ... 'T' 'T' 'G' 'G'

    array(['C', 'C', 'G', ..., 'T', 'G', 'G'], dtype=object)

  • one_allele
    (variant)
    object
    'T' 'T' 'C' 'C' ... 'C' 'C' 'A' 'A'

    array(['T', 'T', 'C', ..., 'C', 'A', 'A'], dtype=object)

  • af
    (variant)
    float64
    nan nan nan nan ... nan nan nan nan

    array([nan, nan, nan, ..., nan, nan, nan])

  • pos_bp
    (variant)
    int32
    157439 157439 ... 27367844 27367844

    array([  157439,   157439,   181802, ..., 27205125, 27367844, 27367844],
          dtype=int32)

  • pos_cM
    (variant)
    float64
    0.0 0.0 0.0 0.0 ... 0.0 0.0 0.0 0.0

    array([0., 0., 0., ..., 0., 0., 0.])

• Indexes: (2)
  • sample
    PandasIndex

    PandasIndex(Index(['0..B001.B001', '0..B002.B002', '0..B003.B003', '0..B004.B004',
           '0..B005.B005', '0..B006.B006', '0..B007.B007', '0..B008.B008',
           '0..B009.B009', '0..B010.B010', '0..B011.B011', '0..B012.B012',
           '0..B013.B013', '0..B014.B014'],
          dtype='object', name='sample'))

  • variant
    PandasIndex

    PandasIndex(Index(['316849996.0', '316849996.1', '316874359.0', '316874359.1',
           '316941526.0', '316941526.1', '317137620.0', '317137620.1',
           '317534352.0', '317534352.1',
           ...
           '372752160.0', '372752160.1', '372819760.0', '372819760.1',
           '372877404.0', '372877404.1', '372918788.0', '372918788.1',
           '373081507.0', '373081507.1'],
          dtype='object', name='variant', length=2504))

• Attributes: (1)

  generation :

  0

As pandas-plink will generate a lazy dask array, we need to use the .compute() method to explicitly load it into memory:

pdat.compute()

<xarray.DataArray 'HaplotypeArray' (sample: 14, variant: 2504)>
array([[0, 0, 0, ..., 1, 0, 0],
       [0, 0, 1, ..., 0, 0, 0],
       [0, 0, 0, ..., 0, 0, 0],
       ...,
       [1, 1, 1, ..., 1, 1, 0],
       [1, 1, 0, ..., 1, 1, 1],
       [0, 0, 0, ..., 0, 0, 0]], dtype=int8)
Coordinates: (12/13)
  * sample       (sample) <U12 '0..B001.B001' '0..B002.B002' ... '0..B014.B014'
    iid          (sample) object 'B001' 'B002' 'B003' ... 'B012' 'B013' 'B014'
    fid          (sample) object 'B001' 'B002' 'B003' ... 'B012' 'B013' 'B014'
    sex          (sample) int64 2 2 2 2 2 2 2 2 2 2 2 2 2 2
  * variant      (variant) <U11 '316849996.0' '316849996.1' ... '373081507.1'
    vid          (variant) object '316849996' '316849996' ... '373081507'
    ...           ...
    hcopy        (variant) object '0' '1' '0' '1' '0' ... '1' '0' '1' '0' '1'
    zero_allele  (variant) object 'C' 'C' 'G' 'G' 'G' ... 'A' 'T' 'T' 'G' 'G'
    one_allele   (variant) object 'T' 'T' 'C' 'C' 'C' ... 'T' 'C' 'C' 'A' 'A'
    af           (variant) float64 nan nan nan nan nan ... nan nan nan nan nan
    pos_bp       (variant) int32 157439 157439 181802 ... 27367844 27367844
    pos_cM       (variant) float64 0.0 0.0 0.0 0.0 0.0 ... 0.0 0.0 0.0 0.0 0.0
Attributes:
    generation:  0

xarray.DataArray

'HaplotypeArray'

• sample: 14
• variant: 2504

• 0 0 0 0 1 1 0 0 0 0 0 0 0 0 0 0 0 ... 0 0 0 0 0 1 1 1 0 0 1 0 0 0 0 0

  array([[0, 0, 0, ..., 1, 0, 0],
         [0, 0, 1, ..., 0, 0, 0],
         [0, 0, 0, ..., 0, 0, 0],
         ...,
         [1, 1, 1, ..., 1, 1, 0],
         [1, 1, 0, ..., 1, 1, 1],
         [0, 0, 0, ..., 0, 0, 0]], dtype=int8)

• Coordinates: (13)
  • sample
    (sample)
    <U12
    '0..B001.B001' ... '0..B014.B014'

    array(['0..B001.B001', '0..B002.B002', '0..B003.B003', '0..B004.B004',
           '0..B005.B005', '0..B006.B006', '0..B007.B007', '0..B008.B008',
           '0..B009.B009', '0..B010.B010', '0..B011.B011', '0..B012.B012',
           '0..B013.B013', '0..B014.B014'], dtype='<U12')

  • iid
    (sample)
    object
    'B001' 'B002' ... 'B013' 'B014'

    array(['B001', 'B002', 'B003', 'B004', 'B005', 'B006', 'B007', 'B008',
           'B009', 'B010', 'B011', 'B012', 'B013', 'B014'], dtype=object)

  • fid
    (sample)
    object
    'B001' 'B002' ... 'B013' 'B014'

    array(['B001', 'B002', 'B003', 'B004', 'B005', 'B006', 'B007', 'B008',
           'B009', 'B010', 'B011', 'B012', 'B013', 'B014'], dtype=object)

  • sex
    (sample)
    int64
    2 2 2 2 2 2 2 2 2 2 2 2 2 2

    array([2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2])

  • variant
    (variant)
    <U11
    '316849996.0' ... '373081507.1'

    array(['316849996.0', '316849996.1', '316874359.0', ..., '372918788.1',
           '373081507.0', '373081507.1'], dtype='<U11')

  • vid
    (variant)
    object
    '316849996' ... '373081507'

    array(['316849996', '316849996', '316874359', ..., '372918788',
           '373081507', '373081507'], dtype=object)

  • chrom
    (variant)
    object
    '11' '11' '11' ... '12' '12' '12'

    array(['11', '11', '11', ..., '12', '12', '12'], dtype=object)

  • hcopy
    (variant)
    object
    '0' '1' '0' '1' ... '0' '1' '0' '1'

    array(['0', '1', '0', ..., '1', '0', '1'], dtype=object)

  • zero_allele
    (variant)
    object
    'C' 'C' 'G' 'G' ... 'T' 'T' 'G' 'G'

    array(['C', 'C', 'G', ..., 'T', 'G', 'G'], dtype=object)

  • one_allele
    (variant)
    object
    'T' 'T' 'C' 'C' ... 'C' 'C' 'A' 'A'

    array(['T', 'T', 'C', ..., 'C', 'A', 'A'], dtype=object)

  • af
    (variant)
    float64
    nan nan nan nan ... nan nan nan nan

    array([nan, nan, nan, ..., nan, nan, nan])

  • pos_bp
    (variant)
    int32
    157439 157439 ... 27367844 27367844

    array([  157439,   157439,   181802, ..., 27205125, 27367844, 27367844],
          dtype=int32)

  • pos_cM
    (variant)
    float64
    0.0 0.0 0.0 0.0 ... 0.0 0.0 0.0 0.0

    array([0., 0., 0., ..., 0., 0., 0.])

• Indexes: (2)
  • sample
    PandasIndex

    PandasIndex(Index(['0..B001.B001', '0..B002.B002', '0..B003.B003', '0..B004.B004',
           '0..B005.B005', '0..B006.B006', '0..B007.B007', '0..B008.B008',
           '0..B009.B009', '0..B010.B010', '0..B011.B011', '0..B012.B012',
           '0..B013.B013', '0..B014.B014'],
          dtype='object', name='sample'))

  • variant
    PandasIndex

    PandasIndex(Index(['316849996.0', '316849996.1', '316874359.0', '316874359.1',
           '316941526.0', '316941526.1', '317137620.0', '317137620.1',
           '317534352.0', '317534352.1',
           ...
           '372752160.0', '372752160.1', '372819760.0', '372819760.1',
           '372877404.0', '372877404.1', '372918788.0', '372918788.1',
           '373081507.0', '373081507.1'],
          dtype='object', name='variant', length=2504))

• Attributes: (1)

  generation :

  0

We can write haplotype arrays out to plink bfiles using xft.io.write_to_plink1():

xft.io.write_to_plink1(pdat, '/tmp/plink')

/home/rsb/Dropbox/ftsim/xftsim/xftsim/io.py:194: UserWarning: Writing to the plink bfile format removes phase information
  warnings.warn("Writing to the plink bfile format removes phase information")
Writing BED: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 202.75it/s]

Writing FAM... done.
Writing BIM... done.

/home/rsb/miniconda3/lib/python3.9/site-packages/pandas_plink/_write.py:261: FutureWarning: the 'line_terminator'' keyword is deprecated, use 'lineterminator' instead.
  df.to_csv(
/home/rsb/miniconda3/lib/python3.9/site-packages/pandas_plink/_write.py:288: FutureWarning: the 'line_terminator'' keyword is deprecated, use 'lineterminator' instead.
  df.to_csv(

VCF / sgkit data

Support for VCF files is provided via compatability the sgkit library. We recommend the following procedure:

1. Convert the VCF file (or a subset there of) to a zarr store on disk using sgkit.io.vcf.vcf_to_zarr()

2. Lazy load the sgkit data using sgkit.load_dataset()

3. Convert the sgkit data to an xftsim compatable DataArray using xft.founders.founder_haplotypes_from_sgkit_dataset()

4. Save the founder data to zarr for future use using xft.io.save_haplotype_zarr()

Haplotypes from zarr stores

Zarr stores are the preferred file format for saving and loading haplotype array data in xftsim. Haplotype data can be read from zarr using xft.io.load_haplotype_zarr() and written using xft.io.save_haplotype_zarr().