Performance tips

Performance tips#

using DataFrames
using BenchmarkTools
using CategoricalArrays
using PooledArrays
using Random

Access by column number is faster than by name#

x = DataFrame(rand(5, 1000), :auto)
@btime $x[!, 500];
@btime $x.x500;

  3.396 ns (0 allocations: 0 bytes)

  10.811 ns (0 allocations: 0 bytes)

When working with data DataFrame use barrier functions or type annotation#

function f_bad() ## this function will be slow
    Random.seed!(1)
    x = DataFrame(rand(1000000, 2), :auto)
    y, z = x[!, 1], x[!, 2]
    p = 0.0
    for i in 1:nrow(x)
        p += y[i] * z[i]
    end
    p
end

@btime f_bad();
# if you run @code_warntype f_bad() then you notice
# that Julia does not know column types of `DataFrame`

  94.707 ms (5999012 allocations: 122.06 MiB)

solution 1 is to use barrier function (it should be possible to use it in almost any code)

function f_inner(y, z)
    p = 0.0
    for i in 1:length(y)
        p += y[i] * z[i]
    end
    p
end

f_inner (generic function with 1 method)

extract the work to an inner function

function f_barrier()
    Random.seed!(1)
    x = DataFrame(rand(1000000, 2), :auto)
    f_inner(x[!, 1], x[!, 2])
end

f_barrier (generic function with 1 method)

or use inbuilt function if possible

using LinearAlgebra
function f_inbuilt()
    Random.seed!(1)
    x = DataFrame(rand(1000000, 2), :auto)
    dot(x[!, 1], x[!, 2])
end

f_inbuilt (generic function with 1 method)

@btime f_barrier();
@btime f_inbuilt();

  4.139 ms (34 allocations: 30.52 MiB)

  3.484 ms (34 allocations: 30.52 MiB)

solution 2 is to provide the types of extracted columns. It is simpler but there are cases in which you will not know these types. This example assumes that you have DataFrames master at least from August 31, 2018

function f_typed()
    Random.seed!(1)
    x = DataFrame(rand(1000000, 2), :auto)
    y::Vector{Float64}, z::Vector{Float64} = x[!, 1], x[!, 2]
    p = 0.0
    for i in 1:nrow(x)
        p += y[i] * z[i]
    end
    p
end

@btime f_typed();

  4.141 ms (34 allocations: 30.52 MiB)

In general for tall and narrow tables it is often useful to use Tables.rowtable, Tables.columntable or Tables.namedtupleiterator for intermediate processing of data in a type-stable way.

Consider using delayed `DataFrame` creation technique#

also notice the difference in performance between copying vs non-copying data frame creation

function f1()
    x = DataFrame([Vector{Float64}(undef, 10^4) for i in 1:100], :auto, copycols=false) ## we work with a DataFrame directly
    for c in 1:ncol(x)
        d = x[!, c]
        for r in 1:nrow(x)
            d[r] = rand()
        end
    end
    x
end

function f1a()
    x = DataFrame([Vector{Float64}(undef, 10^4) for i in 1:100], :auto) ## we work with a DataFrame directly
    for c in 1:ncol(x)
        d = x[!, c]
        for r in 1:nrow(x)
            d[r] = rand()
        end
    end
    x
end

function f2()
    x = Vector{Any}(undef, 100)
    for c in 1:length(x)
        d = Vector{Float64}(undef, 10^4)
        for r in 1:length(d)
            d[r] = rand()
        end
        x[c] = d
    end
    DataFrame(x, :auto, copycols=false) ## we delay creation of DataFrame after we have our job done
end

function f2a()
    x = Vector{Any}(undef, 100)
    for c in 1:length(x)
        d = Vector{Float64}(undef, 10^4)
        for r in 1:length(d)
            d[r] = rand()
        end
        x[c] = d
    end
    DataFrame(x, :auto) ## we delay creation of DataFrame after we have our job done
end

@btime f1();
@btime f1a();
@btime f2();
@btime f2a();

  27.076 ms (1949524 allocations: 37.40 MiB)

  27.883 ms (1949724 allocations: 45.03 MiB)

  1.175 ms (524 allocations: 7.66 MiB)

  1.595 ms (724 allocations: 15.29 MiB)

You can add rows to a DataFrame in place and it is fast#

x = DataFrame(rand(10^6, 5), :auto)
y = DataFrame(transpose(1.0:5.0), :auto)
z = [1.0:5.0;]

@btime vcat($x, $y); ## creates a new DataFrame - slow
@btime append!($x, $y); ## in place - fast

x = DataFrame(rand(10^6, 5), :auto) ## reset to the same starting point
@btime push!($x, $z); ## add a single row in place - fast

  2.820 ms (211 allocations: 38.16 MiB)

  1.371 μs (30 allocations: 1.80 KiB)

  477.405 ns (16 allocations: 256 bytes)

Allowing missing as well as categorical slows down computations#

using StatsBase

function test(data) ## uses countmap function to test performance
    println(eltype(data))
    x = rand(data, 10^6)
    y = categorical(x)
    println(" raw:")
    @btime countmap($x)
    println(" categorical:")
    @btime countmap($y)
    nothing
end

test(1:10)
test([randstring() for i in 1:10])
test(allowmissing(1:10))
test(allowmissing([randstring() for i in 1:10]))

Int64
 raw:

  1.608 ms (7 allocations: 7.63 MiB)
 categorical:

  9.910 ms (4 allocations: 688 bytes)

String
 raw:

  19.638 ms (4 allocations: 528 bytes)
 categorical:

  24.002 ms (4 allocations: 688 bytes)

Union{

Missing, Int64}
 raw:

  7.036 ms (4 allocations: 560 bytes)
 categorical:

  16.008 ms (1000004 allocations: 30.52 MiB)

Union{Missing, String}
 raw:

  19.917 ms (4 allocations: 528 bytes)
 categorical:

  31.779 ms (1000004 allocations: 30.52 MiB)

When aggregating use column selector and prefer integer, categorical, or pooled array grouping variable#

df = DataFrame(x=rand('a':'d', 10^7), y=1);

gdf = groupby(df, :x)

GroupedDataFrame with 4 groups based on key: x

First Group (2501514 rows): x = 'a': ASCII/Unicode U+0061 (category Ll: Letter, lowercase)

2501489 rows omitted

Row	x	y
	Char	Int64
1	a	1
2	a	1
3	a	1
4	a	1
5	a	1
6	a	1
7	a	1
8	a	1
9	a	1
10	a	1
11	a	1
12	a	1
13	a	1
⋮	⋮	⋮
2501503	a	1
2501504	a	1
2501505	a	1
2501506	a	1
2501507	a	1
2501508	a	1
2501509	a	1
2501510	a	1
2501511	a	1
2501512	a	1
2501513	a	1
2501514	a	1

⋮

Last Group (2498598 rows): x = 'd': ASCII/Unicode U+0064 (category Ll: Letter, lowercase)

2498573 rows omitted

Row	x	y
	Char	Int64
1	d	1
2	d	1
3	d	1
4	d	1
5	d	1
6	d	1
7	d	1
8	d	1
9	d	1
10	d	1
11	d	1
12	d	1
13	d	1
⋮	⋮	⋮
2498587	d	1
2498588	d	1
2498589	d	1
2498590	d	1
2498591	d	1
2498592	d	1
2498593	d	1
2498594	d	1
2498595	d	1
2498596	d	1
2498597	d	1
2498598	d	1

traditional syntax, slow

@btime combine(v -> sum(v.y), $gdf)

  15.117 ms (325 allocations: 19.10 MiB)

4×2 DataFrame

Row	x	x1
	Char	Int64
1	a	2501514
2	b	2498412
3	c	2501476
4	d	2498598

use column selector

@btime combine($gdf, :y => sum)

  7.012 ms (186 allocations: 11.45 KiB)

4×2 DataFrame

Row	x	y_sum
	Char	Int64
1	a	2501514
2	b	2498412
3	c	2501476
4	d	2498598

transform!(df, :x => categorical => :x);
gdf = groupby(df, :x)

GroupedDataFrame with 4 groups based on key: x

First Group (2501514 rows): x = CategoricalValue{Char, UInt32} 'a'

2501489 rows omitted

Row	x	y
	Cat…	Int64
1	a	1
2	a	1
3	a	1
4	a	1
5	a	1
6	a	1
7	a	1
8	a	1
9	a	1
10	a	1
11	a	1
12	a	1
13	a	1
⋮	⋮	⋮
2501503	a	1
2501504	a	1
2501505	a	1
2501506	a	1
2501507	a	1
2501508	a	1
2501509	a	1
2501510	a	1
2501511	a	1
2501512	a	1
2501513	a	1
2501514	a	1

⋮

Last Group (2498598 rows): x = CategoricalValue{Char, UInt32} 'd'

2498573 rows omitted

Row	x	y
	Cat…	Int64
1	d	1
2	d	1
3	d	1
4	d	1
5	d	1
6	d	1
7	d	1
8	d	1
9	d	1
10	d	1
11	d	1
12	d	1
13	d	1
⋮	⋮	⋮
2498587	d	1
2498588	d	1
2498589	d	1
2498590	d	1
2498591	d	1
2498592	d	1
2498593	d	1
2498594	d	1
2498595	d	1
2498596	d	1
2498597	d	1
2498598	d	1

@btime combine($gdf, :y => sum)

  6.969 ms (193 allocations: 12.03 KiB)

4×2 DataFrame

Row	x	y_sum
	Cat…	Int64
1	a	2501514
2	b	2498412
3	c	2501476
4	d	2498598

transform!(df, :x => PooledArray{Char} => :x)

10000000×2 DataFrame

9999975 rows omitted

Row	x	y
	Char	Int64
1	a	1
2	a	1
3	a	1
4	b	1
5	a	1
6	c	1
7	d	1
8	d	1
9	b	1
10	c	1
11	b	1
12	b	1
13	c	1
⋮	⋮	⋮
9999989	b	1
9999990	a	1
9999991	b	1
9999992	d	1
9999993	d	1
9999994	a	1
9999995	d	1
9999996	d	1
9999997	b	1
9999998	d	1
9999999	b	1
10000000	b	1

gdf = groupby(df, :x)

GroupedDataFrame with 4 groups based on key: x

First Group (2501514 rows): x = 'a': ASCII/Unicode U+0061 (category Ll: Letter, lowercase)

2501489 rows omitted

Row	x	y
	Char	Int64
1	a	1
2	a	1
3	a	1
4	a	1
5	a	1
6	a	1
7	a	1
8	a	1
9	a	1
10	a	1
11	a	1
12	a	1
13	a	1
⋮	⋮	⋮
2501503	a	1
2501504	a	1
2501505	a	1
2501506	a	1
2501507	a	1
2501508	a	1
2501509	a	1
2501510	a	1
2501511	a	1
2501512	a	1
2501513	a	1
2501514	a	1

⋮

Last Group (2498598 rows): x = 'd': ASCII/Unicode U+0064 (category Ll: Letter, lowercase)

2498573 rows omitted

Row	x	y
	Char	Int64
1	d	1
2	d	1
3	d	1
4	d	1
5	d	1
6	d	1
7	d	1
8	d	1
9	d	1
10	d	1
11	d	1
12	d	1
13	d	1
⋮	⋮	⋮
2498587	d	1
2498588	d	1
2498589	d	1
2498590	d	1
2498591	d	1
2498592	d	1
2498593	d	1
2498594	d	1
2498595	d	1
2498596	d	1
2498597	d	1
2498598	d	1

@btime combine($gdf, :y => sum)

  7.009 ms (188 allocations: 11.52 KiB)

4×2 DataFrame

Row	x	y_sum
	Char	Int64
1	a	2501514
2	b	2498412
3	c	2501476
4	d	2498598

Use views instead of materializing a new DataFrame#

x = DataFrame(rand(100, 1000), :auto)

100×1000 DataFrame

900 columns and 75 rows omitted

Row	x1	x2	x3	x4	x5	x6	x7	x8	x9	x10	x11	x12	x13	x14	x15	x16	x17	x18	x19	x20	x21	x22	x23	x24	x25	x26	x27	x28	x29	x30	x31	x32	x33	x34	x35	x36	x37	x38	x39	x40	x41	x42	x43	x44	x45	x46	x47	x48	x49	x50	x51	x52	x53	x54	x55	x56	x57	x58	x59	x60	x61	x62	x63	x64	x65	x66	x67	x68	x69	x70	x71	x72	x73	x74	x75	x76	x77	x78	x79	x80	x81	x82	x83	x84	x85	x86	x87	x88	x89	x90	x91	x92	x93	x94	x95	x96	x97	x98	x99	x100	⋯
	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	⋯
1	0.855908	0.999016	0.790408	0.268832	0.495286	0.749496	0.333178	0.0277931	0.352712	0.329304	0.691565	0.069195	0.796135	0.952	0.73932	0.328348	0.734515	0.126988	0.341958	0.935508	0.282837	0.877078	0.84413	0.870527	0.502392	0.0219476	0.73117	0.242422	0.0977879	0.730115	0.55251	0.654095	0.640858	0.838408	0.510724	0.0346715	0.136903	0.249659	0.273487	0.232657	0.957472	0.844805	0.812701	0.589498	0.0508585	0.688785	0.969166	0.847723	0.681458	0.36317	0.246428	0.875776	0.18572	0.200951	0.684077	0.727957	0.560089	0.759244	0.045221	0.54017	0.110852	0.537911	0.934418	0.573776	0.365234	0.250545	0.863869	0.108769	0.0556473	0.690457	0.752701	0.301829	0.0969686	0.996922	0.596123	0.972959	0.329478	0.811853	0.396499	0.956983	0.319399	0.0920556	0.986654	0.175337	0.569408	0.773597	0.134914	0.889006	0.120567	0.477122	0.678824	0.711837	0.0697078	0.120692	0.532224	0.275187	0.801821	0.767578	0.616763	0.556994	⋯
2	0.0518128	0.431624	0.861054	0.253188	0.00920565	0.152964	0.821862	0.599065	0.493643	0.865247	0.112971	0.534441	0.280611	0.140355	0.276637	0.811366	0.810516	0.745645	0.870767	0.356638	0.881723	0.511726	0.43061	0.576055	0.0750177	0.683997	0.606695	0.564738	0.141649	0.582117	0.587465	0.379532	0.653079	0.0599559	0.0610893	0.780742	0.290271	0.266464	0.817084	0.147441	0.369141	0.782436	0.785157	0.93323	0.363198	0.301652	0.094921	0.860451	0.600444	0.254886	0.930544	0.620389	0.85234	0.504318	0.498758	0.425582	0.220443	0.494531	0.999702	0.532573	0.479343	0.126767	0.30973	0.3948	0.69753	0.260657	0.898102	0.782046	0.0806481	0.365754	0.269173	0.341762	0.666861	0.480426	0.155309	0.941425	0.869729	0.881828	0.756368	0.58615	0.797519	0.261844	0.810964	0.0577036	0.282548	0.64299	0.285154	0.0511854	0.937673	0.0165954	0.871602	0.290727	0.185748	0.645132	0.455692	0.744877	0.398908	0.235863	0.0172382	0.157506	⋯
3	0.898662	0.717985	0.252469	0.68483	0.371952	0.0963873	0.176943	0.929472	0.192254	0.605792	0.648397	0.531858	0.700302	0.0408464	0.772701	0.651476	0.375052	0.487822	0.866109	0.138173	0.00299112	0.772063	0.715147	0.0131052	0.555178	0.861791	0.23716	0.578593	0.45746	0.38744	0.533027	0.317145	0.885861	0.39084	0.777834	0.503903	0.80463	0.0115711	0.519663	0.630738	0.621413	0.915116	0.181002	0.113939	0.0244349	0.553214	0.537904	0.998062	0.00200446	0.434135	0.0844599	0.263396	0.541131	0.793399	0.748969	0.497816	0.689179	0.602533	0.875838	0.696204	0.209088	0.78793	0.59816	0.857832	0.598175	0.736086	0.672982	0.921685	0.117355	0.821865	0.642697	0.691033	0.580495	0.500364	0.0348166	0.0589426	0.761911	0.788373	0.338558	0.299761	0.597552	0.617057	0.744483	0.269249	0.859521	0.634367	0.678527	0.530986	0.341439	0.671747	0.536034	0.727509	0.0485432	0.618044	0.181725	0.617929	0.736747	0.0147766	0.689982	0.928152	⋯
4	0.945066	0.159603	0.296586	0.355634	0.806893	0.240168	0.0474817	0.203433	0.951745	0.595304	0.156072	0.467918	0.212253	0.661345	0.0403789	0.650948	0.401636	0.944865	0.809386	0.144122	0.805034	0.527208	0.739797	0.192907	0.984467	0.85686	0.696416	0.0904573	0.0149675	0.61515	0.538791	0.0396014	0.110696	0.86863	0.624317	0.150555	0.648641	0.438351	0.291852	0.960628	0.146095	0.715015	0.776477	0.540114	0.563072	0.979048	0.677097	0.587737	0.560573	0.644031	0.598253	0.362407	0.846621	0.367037	0.901299	0.412503	0.948126	0.564	0.857078	0.501544	0.283434	0.872934	0.811785	0.988503	0.811275	0.55343	0.993947	0.128693	0.910252	0.210555	0.0624667	0.285734	0.60173	0.872546	0.367605	0.818695	0.380678	0.764841	0.251535	0.400985	0.869971	0.487101	0.398209	0.226695	0.817518	0.251888	0.231154	0.546388	0.480906	0.365457	0.660366	0.945626	0.640348	0.269014	0.606143	0.185614	0.87125	0.609309	0.0731741	0.696282	⋯
5	0.991737	0.518838	0.626403	0.0149906	0.360394	0.0047324	0.302574	0.83853	0.300462	0.15674	0.746388	0.190238	0.0221994	0.515072	0.154438	0.0482817	0.363129	0.934825	0.138421	0.716409	0.411773	0.757526	0.407278	0.199432	0.386159	0.299488	0.893673	0.813782	0.0263761	0.0946466	0.196759	0.664092	0.526779	0.274572	0.0749945	0.989309	0.684761	0.847977	0.240271	0.85787	0.852323	0.105957	0.39709	0.901893	0.15148	0.0130823	0.226468	0.318715	0.543911	0.602061	0.881846	0.117082	0.309799	0.854567	0.0378424	0.298761	0.923345	0.321644	0.0764862	0.036251	0.686	0.82489	0.123272	0.942327	0.948223	0.171074	0.520013	0.117832	0.0331662	0.655575	0.425343	0.964315	0.422135	0.915215	0.375715	0.664895	0.308244	0.015342	0.0929499	0.476017	0.922516	0.146256	0.954482	0.366822	0.716765	0.266405	0.650361	0.876923	0.530313	0.437883	0.322044	0.0840408	0.655216	0.324763	0.889709	0.192837	0.231169	0.790742	0.317069	0.226672	⋯
6	0.458762	0.120676	0.553207	0.875395	0.496451	0.882567	0.896253	0.127758	0.430508	0.702534	0.134364	0.458922	0.252057	0.466628	0.559586	0.382155	0.159453	0.596648	0.363884	0.123607	0.936625	0.607611	0.079235	0.887775	0.799588	0.163187	0.628142	0.482754	0.229375	0.66565	0.717323	0.79415	0.754256	0.482791	0.841862	0.403774	0.376718	0.186009	0.0956262	0.847502	0.0875078	0.455088	0.425437	0.937393	0.783907	0.505973	0.215259	0.600183	0.978893	0.208397	0.142225	0.790035	0.428786	0.257695	0.830765	0.740989	0.238608	0.801573	0.0858531	0.0734213	0.801522	0.292001	0.571271	0.703333	0.559965	0.613792	0.0861506	0.0821297	0.765177	0.618307	0.298734	0.856788	0.549248	0.406026	0.583994	0.349411	0.516085	0.188554	0.226328	0.715651	0.0198637	0.846588	0.753127	0.834075	0.620492	0.362903	0.245742	0.463876	0.628169	0.468889	0.96262	0.73566	0.995707	0.937926	0.253771	0.692229	0.313781	0.917841	0.559493	0.00182348	⋯
7	0.897691	0.91969	0.780247	0.697717	0.490213	0.556836	0.891886	0.823419	0.2019	0.258225	0.869342	0.292135	0.968302	0.823566	0.677918	0.27533	0.928954	0.153419	0.456327	0.194386	0.305116	0.594922	0.123877	0.73576	0.884764	0.0382912	0.301087	0.562251	0.475792	0.475493	0.513928	0.751546	0.703126	0.29589	0.585073	0.890962	0.134772	0.198431	0.511007	0.0799777	0.139068	0.351919	0.48634	0.564159	0.519392	0.475552	0.0791789	0.220919	0.388412	0.779677	0.437774	0.962808	0.71542	0.99454	0.205762	0.443139	0.530181	0.160694	0.970132	0.123498	0.0754974	0.92929	0.737945	0.911881	0.653813	0.693283	0.411188	0.21774	0.717303	0.192665	0.431551	0.510125	0.349035	0.641758	0.939971	0.592723	0.860791	0.693388	0.886439	0.117239	0.572271	0.0930164	0.581306	0.365233	0.403769	0.649345	0.344614	0.9	0.883324	0.352412	0.516188	0.140929	0.192595	0.989137	0.208372	0.336612	0.272344	0.170864	0.595507	0.025991	⋯
8	0.360176	0.184166	0.702679	0.642936	0.494688	0.149384	0.567358	0.239442	0.828555	0.312631	0.323533	0.40291	0.841792	0.922335	0.881522	0.390515	0.43772	0.606186	0.309979	0.926584	0.254194	0.524438	0.556385	0.976477	0.75672	0.313465	0.714924	0.838343	0.418008	0.808185	0.181491	0.512642	0.685151	0.210081	0.417746	0.420704	0.560027	0.350075	0.418959	0.949772	0.941496	0.532926	0.714375	0.218629	0.162552	0.0498728	0.533168	0.833329	0.603833	0.0979217	0.613309	0.951886	0.321275	0.345883	0.5004	0.258303	0.764755	0.357824	0.957521	0.0492376	0.417887	0.112941	0.650659	0.965046	0.855573	0.598635	0.908266	0.759521	0.958749	0.406882	0.822446	0.769839	0.248723	0.00638755	0.955277	0.989699	0.333705	0.731751	0.683357	0.918864	0.360642	0.372474	0.168796	0.267602	0.528234	0.731622	0.848043	0.895275	0.127814	0.980725	0.283604	0.28004	0.993089	0.836196	0.471962	0.148876	0.90317	0.74567	0.181602	0.759762	⋯
9	0.776218	0.781912	0.610356	0.607579	0.309328	0.777644	0.522816	0.474342	0.716939	0.0463905	0.127173	0.841067	0.739222	0.97784	0.72954	0.35684	0.409911	0.0309987	0.515681	0.145383	0.978123	0.0436661	0.727387	0.470658	0.96789	0.343701	0.767133	0.494749	0.666689	0.256223	0.197891	0.783577	0.3344	0.580823	0.445869	0.521965	0.986471	0.71381	0.779094	0.928658	0.654714	0.154975	0.219946	0.286612	0.857774	0.341746	0.0029352	0.881218	0.520579	0.985804	0.695808	0.00685713	0.421405	0.188353	0.948122	0.57991	0.499082	0.164843	0.150474	0.0509632	0.858247	0.772151	0.284107	0.121437	0.908849	0.0138105	0.660202	0.421663	0.62901	0.536077	0.946827	0.334471	0.620629	0.710756	0.51965	0.231777	0.802411	0.556794	0.26502	0.70461	0.298894	0.164058	0.314167	0.000809659	0.878456	0.936473	0.115144	0.978186	0.50314	0.620725	0.260309	0.729319	0.49595	0.169431	0.286847	0.455739	0.85666	0.172033	0.115669	0.834543	⋯
10	0.111145	0.751312	0.0822745	0.912968	0.674156	0.870846	0.465226	0.701494	0.972352	0.728111	0.579633	0.0925833	0.626908	0.086513	0.172604	0.351198	0.580904	0.432892	0.180149	0.623499	0.428795	0.748824	0.144205	0.842455	0.0954633	0.295822	0.17516	0.231135	0.351148	0.649209	0.940991	0.366495	0.529277	0.923593	0.866123	0.957947	0.981533	0.712121	0.109842	0.126495	0.530302	0.216533	0.0198751	0.801459	0.69886	0.780993	0.754113	0.0315909	0.239606	0.607388	0.90564	0.319761	0.544419	0.553326	0.132488	0.195052	0.461764	0.383799	0.518886	0.0525735	0.57942	0.52158	0.776768	0.638295	0.649076	0.443292	0.855524	0.985679	0.909259	0.14368	0.456238	0.946942	0.371847	0.669677	0.497083	0.446804	0.985571	0.900206	0.74769	0.609282	0.579441	0.106572	0.656446	0.695381	0.752143	0.313355	0.97009	0.635052	0.135057	0.359388	0.45726	0.988183	0.971734	0.711609	0.298734	0.687311	0.559806	0.911289	0.0464569	0.35505	⋯
11	0.119991	0.132612	0.0154578	0.393048	0.989334	0.330142	0.828721	0.510056	0.164207	0.335458	0.348903	0.579522	0.832099	0.804131	0.341627	0.477059	0.0599161	0.425	0.259572	0.239568	0.965023	0.711486	0.914983	0.719582	0.590793	0.0538001	0.718897	0.819057	0.0620284	0.278699	0.973708	0.168269	0.0642724	0.333387	0.271288	0.764147	0.0127272	0.0849106	0.409811	0.593149	0.683857	0.345843	0.318101	0.858299	0.343922	0.149796	0.644013	0.691998	0.253213	0.365093	0.759355	0.150212	0.0749318	0.587274	0.832099	0.474888	0.208505	0.621962	0.484155	0.867005	0.249235	0.276332	0.39219	0.971576	0.594608	0.226268	0.662043	0.391796	0.0424286	0.819572	0.948792	0.554172	0.402852	0.305945	0.00523565	0.041456	0.889241	0.336184	0.968253	0.773377	0.956724	0.490755	0.354714	0.605031	0.724376	0.512901	0.876489	0.0775624	0.877199	0.78544	0.213787	0.756972	0.429244	0.449597	0.143313	0.470965	0.943779	0.269517	0.0525089	0.592346	⋯
12	0.283766	0.51444	0.149057	0.128911	0.248536	0.218714	0.79223	0.166148	0.600264	0.383417	0.670591	0.631413	0.457213	0.411288	0.955439	0.194741	0.88413	0.617257	0.0664118	0.925707	0.169628	0.210315	0.659738	0.9803	0.0272076	0.765862	0.60712	0.535387	0.350023	0.9152	0.949086	0.975962	0.732657	0.291345	0.30687	0.836892	0.73154	0.569059	0.526566	0.934957	0.389851	0.451778	0.675706	0.503968	0.0598252	0.591706	0.505422	0.414773	0.168351	0.159409	0.666413	0.246388	0.0471972	0.288563	0.773062	0.138127	0.331065	0.506016	0.478711	0.36237	0.000818554	0.0432103	0.545035	0.32702	0.65394	0.415736	0.621449	0.44902	0.920662	0.946022	0.336788	0.650228	0.646329	0.905881	0.690546	0.479468	0.823996	0.786648	0.723918	0.122369	0.294296	0.140019	0.96399	0.420845	0.821308	0.543036	0.421838	0.964282	0.913352	0.344111	0.488473	0.553493	0.360206	0.317806	0.635647	0.966033	0.713109	0.132881	0.138337	0.529685	⋯
13	0.629387	0.789667	0.864262	0.334337	0.104678	0.780834	0.676643	0.474721	0.875105	0.697787	0.266335	0.894333	0.639986	0.942782	0.69828	0.266487	0.454184	0.628663	0.029688	0.936807	0.284604	0.235232	0.951609	0.0044829	0.0259861	0.0880318	0.298022	0.371957	0.924782	0.392175	0.277099	0.566338	0.743314	0.172521	0.312054	0.305909	0.617893	0.429058	0.183265	0.0566214	0.722311	0.30022	0.498631	0.190455	0.604297	0.400554	0.75731	0.437659	0.648445	0.901814	0.207004	0.409334	0.78386	0.164186	0.984614	0.87622	0.495325	0.81519	0.0411413	0.124657	0.525325	0.506371	0.186797	0.463646	0.504206	0.00524307	0.618177	0.00893334	0.472995	0.0341628	0.232541	0.248216	0.645995	0.686218	0.762422	0.896391	0.37218	0.127893	0.394156	0.179107	0.609496	0.82726	0.0361556	0.182493	0.101578	0.300017	0.24017	0.839681	0.857251	0.269995	0.746053	0.344779	0.194907	0.157608	0.900688	0.592429	0.234639	0.398257	0.474994	0.435093	⋯
⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋱
89	0.199524	0.820385	0.76606	0.681354	0.987537	0.816368	0.132167	0.598368	0.0335732	0.65481	0.219882	0.825854	0.419101	0.420406	0.446574	0.875308	0.363291	0.447722	0.484297	0.558969	0.161117	0.203264	0.959061	0.0410372	0.231661	0.737196	0.943963	0.86398	0.279363	0.312797	0.304401	0.0464406	0.347023	0.528089	0.787486	0.809677	0.230327	0.0942288	0.16021	0.892883	0.06135	0.961422	0.337225	0.0718496	0.692407	0.393642	0.732449	0.0713918	0.18663	0.639482	0.85842	0.51394	0.639263	0.906459	0.764278	0.316966	0.0704483	0.769569	0.780463	0.666048	0.846227	0.762419	0.983165	0.948482	0.158435	0.818008	0.204003	0.273567	0.669224	0.704182	0.780685	0.302888	0.0201381	0.743193	0.425073	0.0029281	0.725331	0.714512	0.793895	0.306366	0.802565	0.456098	0.119124	0.555749	0.133569	0.873096	0.419136	0.49887	0.674093	0.912343	0.831582	0.0506423	0.214743	0.183765	0.285215	0.672562	0.617899	0.0302556	0.218891	0.991681	⋯
90	0.721207	0.496369	0.141437	0.777523	0.242931	0.436129	0.995043	0.797912	0.0436293	0.114707	0.166887	0.174109	0.450837	0.633731	0.128312	0.438567	0.345595	0.990802	0.695748	0.437228	0.222825	0.941507	0.412085	0.892873	0.377098	0.538597	0.29944	0.935324	0.936007	0.250167	0.621654	0.905371	0.979213	0.291535	0.658584	0.48441	0.113853	0.510823	0.285669	0.904674	0.496955	0.412748	0.755524	0.279045	0.178158	0.434841	0.824064	0.86875	0.822848	0.0706372	0.574235	0.268368	0.240654	0.519585	0.263991	0.820692	0.716553	0.956289	0.895283	0.117812	0.209492	0.791349	0.560794	0.35268	0.736583	0.689648	0.72356	0.623755	0.7927	0.595082	0.0623499	0.822702	0.443735	0.792534	0.721975	0.685356	0.327742	0.562009	0.733137	0.605472	0.465568	0.145934	0.226849	0.949501	0.318389	0.180278	0.238785	0.622927	0.757668	0.838389	0.205891	0.782303	0.647258	0.248851	0.390781	0.323968	0.134091	0.139325	0.427892	0.72248	⋯
91	0.429551	0.573013	0.980245	0.232936	0.178164	0.123522	0.899785	0.478107	0.463129	0.690542	0.311045	0.0879508	0.411091	0.733257	0.909767	0.217972	0.659679	0.799638	0.723674	0.433412	0.638927	0.110125	0.0845372	0.314643	0.387526	0.903415	0.600362	0.623904	0.728458	0.449204	0.780847	0.501676	0.898696	0.543742	0.0354189	0.518394	0.62409	0.487667	0.284078	0.527805	0.61239	0.97219	0.388591	0.785731	0.334676	0.500929	0.926791	0.292833	0.235551	0.119177	0.997722	0.451445	0.910737	0.974942	0.173936	0.671121	0.732212	0.887085	0.756815	0.921579	0.390455	0.99934	0.632062	0.271911	0.18795	0.279628	0.0801043	0.192704	0.279632	0.963021	0.613211	0.49838	0.83057	0.0545646	0.586433	0.0785306	0.187665	0.525095	0.994407	0.424262	0.365687	0.346457	0.198335	0.0513235	0.42533	0.54689	0.677241	0.232787	0.907036	0.527982	0.276876	0.0953511	0.449398	0.356515	0.222586	0.370955	0.849681	0.552147	0.0225842	0.0491013	⋯
92	0.328722	0.756091	0.730735	0.805392	0.699899	0.507488	0.200874	0.904997	0.300451	0.699206	0.138711	0.47792	0.293918	0.27103	0.784963	0.852222	0.54265	0.955793	0.593399	0.979212	0.746245	0.496993	0.0957858	0.133105	0.25713	0.571041	0.646406	0.452294	0.231872	0.0430722	0.0977462	0.948335	0.896083	0.945432	0.0754492	0.175094	0.85313	0.246827	0.901702	0.833706	0.980096	0.693744	0.922955	0.853866	0.268549	0.942425	0.820294	0.20909	0.396905	0.948422	0.482528	0.516679	0.259711	0.091132	0.189758	0.6414	0.19791	0.825063	0.769561	0.906475	0.823828	0.546229	0.749791	0.199471	0.606655	0.654126	0.218732	0.221448	0.429832	0.541889	0.611005	0.548422	0.16064	0.235069	0.42866	0.354419	0.489311	0.0164325	0.160622	0.410974	0.692512	0.203204	0.909168	0.833099	0.94746	0.364444	0.557203	0.765445	0.907459	0.980366	0.542725	0.318987	0.756362	0.685175	0.250173	0.488801	0.671905	0.321592	0.550712	0.470534	⋯
93	0.196454	0.291986	0.189673	0.590915	0.215799	0.491637	0.112798	0.996537	0.431797	0.987754	0.475539	0.452205	0.47431	0.302137	0.0186024	0.27108	0.0898691	0.359314	0.0798957	0.379363	0.445578	0.506911	0.945927	0.52923	0.768044	0.253094	0.0558777	0.823288	0.732035	0.694889	0.409774	0.12972	0.873704	0.850734	0.660076	0.216421	0.578801	0.07892	0.929827	0.20473	0.115393	0.0790629	0.381747	0.206065	0.885686	0.0670902	0.576823	0.468087	0.234705	0.218372	0.569141	0.699547	0.557395	0.803198	0.779086	0.0681689	0.313692	0.461592	0.737543	0.456558	0.722648	0.581549	0.32565	0.985226	0.688996	0.565003	0.329003	0.858703	0.974918	0.260445	0.0432573	0.213001	0.125962	0.660474	0.494476	0.105422	0.557041	0.328293	0.760524	0.928704	0.155504	0.697449	0.184865	0.469322	0.901627	0.936783	0.0696402	0.0970435	0.636556	0.652558	0.586096	0.740089	0.502758	0.207797	0.117115	0.836955	0.104772	0.556371	0.473573	0.574992	⋯
94	0.698035	0.113827	0.984051	0.00234121	0.284129	0.879077	0.371918	0.35856	0.550305	0.120527	0.467733	0.269486	0.784399	0.253837	0.79954	0.808061	0.573156	0.45537	0.648763	0.847761	0.299301	0.940455	0.574567	0.0100698	0.65089	0.517864	0.987935	0.453016	0.839344	0.79812	0.161542	0.459738	0.012988	0.571121	0.210128	0.956155	0.512686	0.377522	0.165742	0.0343293	0.636157	0.582607	0.288329	0.897072	0.532153	0.397285	0.550235	0.41996	0.372448	0.535667	0.110872	0.495717	0.966036	0.795392	0.877696	0.316454	0.0518711	0.158524	0.42255	0.906868	0.12485	0.181697	0.177821	0.716802	0.0542199	0.279458	0.850434	0.316262	0.541499	0.648506	0.765044	0.445916	0.384603	0.89876	0.804104	0.152408	0.212454	0.859118	0.46732	0.777326	0.0520797	0.568455	0.722224	0.374296	0.542193	0.422036	0.42957	0.700292	0.667487	0.014826	0.966193	0.440988	0.74777	0.218231	0.675302	0.289224	0.112795	0.243928	0.67654	0.58545	⋯
95	0.838072	0.873507	0.489406	0.63521	0.27397	0.901083	0.511085	0.11531	0.491743	0.392406	0.0365656	0.492167	0.347453	0.445071	0.187106	0.0994554	0.916556	0.76522	0.0419048	0.937446	0.967987	0.00902701	0.00677831	0.964126	0.874232	0.346421	0.42253	0.461553	0.44168	0.544078	0.538925	0.592894	0.752045	0.928043	0.233629	0.0264467	0.0535002	0.187027	0.558267	0.572397	0.962203	0.00829911	0.486552	0.902428	0.880875	0.391668	0.613384	0.318147	0.612006	0.0832919	0.260875	0.437954	0.2683	0.953675	0.678393	0.320096	0.834402	0.868064	0.81844	0.31808	0.361289	0.750043	0.509331	0.425475	0.0896388	0.787129	0.534697	0.650184	0.0365325	0.173828	0.424635	0.484744	0.577428	0.494511	0.496484	0.711512	0.84783	0.855489	0.489508	0.883316	0.525872	0.394757	0.944976	0.979743	0.175553	0.675713	0.824195	0.378729	0.329007	0.380373	0.311348	0.896653	0.497692	0.803168	0.205711	0.409082	0.680443	0.809838	0.526443	0.358172	⋯
96	0.650231	0.299872	0.244961	0.901608	0.978347	0.382229	0.518668	0.735389	0.560321	0.243305	0.990014	0.491926	0.980999	0.359438	0.316018	0.821982	0.173055	0.993522	0.562082	0.523843	0.582756	0.139051	0.654171	0.617709	0.357705	0.492492	0.195676	0.823245	0.0399039	0.63685	0.347598	0.909822	0.206514	0.407095	0.693153	0.628122	0.214674	0.578234	0.388173	0.0142965	0.708478	0.631085	0.206404	0.698665	0.802772	0.65914	0.672738	0.821893	0.565744	0.51621	0.716372	0.960599	0.64153	0.916871	0.601424	0.31586	0.506618	0.914731	0.407481	0.399059	0.050711	0.519686	0.320552	0.863725	0.331222	0.114251	0.911815	0.177067	0.460959	0.710193	0.394839	0.181673	0.484609	0.196832	0.906602	0.28447	0.460526	0.228945	0.546581	0.708841	0.703991	0.907978	0.301333	0.647407	0.666373	0.767586	0.321271	0.201592	0.0235385	0.542855	0.260629	0.377957	0.830695	0.49322	0.215521	0.256457	0.412514	0.740475	0.0218462	0.873768	⋯
97	0.500948	0.577076	0.72209	0.318255	0.979942	0.127138	0.901253	0.942862	0.66413	0.287497	0.355537	0.648575	0.232757	0.751885	0.142878	0.859228	0.0926723	0.587735	0.339379	0.193896	0.303921	0.188119	0.691497	0.797263	0.964262	0.82456	0.985764	0.280129	0.191846	0.642211	0.0556045	0.83223	0.0015423	0.12899	0.350296	0.774507	0.734476	0.725192	0.0983288	0.364447	0.938871	0.878359	0.599105	0.259483	0.182843	0.466391	0.603177	0.152242	0.0716316	0.057282	0.726159	0.94478	0.498669	0.106939	0.456012	0.834931	0.46035	0.386632	0.645429	0.592436	0.65817	0.898082	0.898496	0.02912	0.777825	0.768493	0.956083	0.606962	0.526577	0.457173	0.117457	0.692526	0.535544	0.807495	0.406067	0.84128	0.651238	0.891348	0.625585	0.648417	0.526726	0.386285	0.875086	0.596622	0.718367	0.912503	0.666084	0.159038	0.296241	0.915779	0.971244	0.428951	0.276462	0.0561139	0.628473	0.562845	0.24762	0.997375	0.840758	0.664162	⋯
98	0.428124	0.600688	0.857956	0.837394	0.889229	0.339414	0.692119	0.51179	0.556134	0.845009	0.391326	0.66287	0.925256	0.925202	0.749067	0.449969	0.0218025	0.176024	0.976721	0.711818	0.52612	0.222535	0.943277	0.897648	0.569348	0.537357	0.806347	0.502458	0.611352	0.458065	0.179212	0.130276	0.84853	0.266908	0.97457	0.32562	0.171563	0.831479	0.387195	0.282175	0.13004	0.236723	0.360267	0.544842	0.608265	0.176241	0.956765	0.407269	0.776842	0.0678049	0.735627	0.134582	0.305341	0.69381	0.276236	0.783877	0.272586	0.568762	0.353748	0.366438	0.0343724	0.93486	0.646909	0.581368	0.0305751	0.209425	0.878409	0.441539	0.964266	0.704148	0.514177	0.661194	0.528893	0.816894	0.065725	0.604232	0.935587	0.432803	0.306891	0.549348	0.638194	0.666597	0.600048	0.693647	0.594923	0.487371	0.669944	0.0478456	0.00457661	0.42912	0.974485	0.991313	0.361193	0.759976	0.537461	0.501231	0.615886	0.605742	0.289823	0.393555	⋯
99	0.80689	0.0376449	0.529051	0.0652305	0.868657	0.18816	0.334661	0.063481	0.518134	0.726146	0.557393	0.780664	0.495658	0.336547	0.35199	0.692252	0.319031	0.159006	0.363338	0.465614	0.202489	0.970108	0.413587	0.464534	0.558864	0.0784824	0.277259	0.0169747	0.52681	0.0826336	0.363325	0.0393903	0.635492	0.597595	0.2419	0.150743	0.752887	0.310077	0.0719189	0.936444	0.0953152	0.711723	0.513305	0.124888	0.308154	0.211424	0.487814	0.384675	0.327147	0.746479	0.129069	0.524325	0.974012	0.964623	0.62342	0.0735421	0.442492	0.324939	0.309651	0.760046	0.0245326	0.452867	0.261891	0.709603	0.594127	0.355437	0.490069	0.763169	0.337695	0.519045	0.917712	0.92382	0.424723	0.0453863	0.994322	0.299885	0.440464	0.426679	0.201862	0.919437	0.697099	0.599743	0.00778926	0.210822	0.324819	0.633669	0.295099	0.808608	0.193917	0.484496	0.390655	0.729434	0.488427	0.183348	0.264667	0.404729	0.122604	0.00282889	0.912303	0.882001	⋯
100	0.529637	0.681621	0.663547	0.709302	0.951104	0.596945	0.31168	0.891946	0.932908	0.341637	0.226385	0.230384	0.0407744	0.166811	0.0128362	0.681319	0.222108	0.655202	0.0393856	0.784179	0.714975	0.615542	0.475701	0.343129	0.912708	0.924669	0.100873	0.41411	0.579827	0.579831	0.699957	0.14158	0.170579	0.587778	0.923107	0.900319	0.941953	0.714221	0.701876	0.530226	0.775462	0.478591	0.852771	0.667887	0.689796	0.00812014	0.414474	0.777172	0.291419	0.157403	0.791543	0.0415275	0.663099	0.539681	0.964152	0.79947	0.898343	0.83411	0.112908	0.805401	0.408063	0.835481	0.583716	0.220022	0.914198	0.552499	0.306172	0.134689	0.58055	0.559835	0.554971	0.875235	0.58892	0.543193	0.502163	0.36637	0.650427	0.410911	0.812828	0.080191	0.714688	0.908417	0.660916	0.583141	0.824269	0.164041	0.390726	0.00339404	0.45995	0.78823	0.564789	0.301719	0.00962449	0.969249	0.886454	0.42111	0.263129	0.199	0.430545	0.532452	⋯

@btime $x[1:1, :]

  186.699 μs (2986 allocations: 159.50 KiB)

1×1000 DataFrame

900 columns omitted

Row	x1	x2	x3	x4	x5	x6	x7	x8	x9	x10	x11	x12	x13	x14	x15	x16	x17	x18	x19	x20	x21	x22	x23	x24	x25	x26	x27	x28	x29	x30	x31	x32	x33	x34	x35	x36	x37	x38	x39	x40	x41	x42	x43	x44	x45	x46	x47	x48	x49	x50	x51	x52	x53	x54	x55	x56	x57	x58	x59	x60	x61	x62	x63	x64	x65	x66	x67	x68	x69	x70	x71	x72	x73	x74	x75	x76	x77	x78	x79	x80	x81	x82	x83	x84	x85	x86	x87	x88	x89	x90	x91	x92	x93	x94	x95	x96	x97	x98	x99	x100	⋯
	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	⋯
1	0.855908	0.999016	0.790408	0.268832	0.495286	0.749496	0.333178	0.0277931	0.352712	0.329304	0.691565	0.069195	0.796135	0.952	0.73932	0.328348	0.734515	0.126988	0.341958	0.935508	0.282837	0.877078	0.84413	0.870527	0.502392	0.0219476	0.73117	0.242422	0.0977879	0.730115	0.55251	0.654095	0.640858	0.838408	0.510724	0.0346715	0.136903	0.249659	0.273487	0.232657	0.957472	0.844805	0.812701	0.589498	0.0508585	0.688785	0.969166	0.847723	0.681458	0.36317	0.246428	0.875776	0.18572	0.200951	0.684077	0.727957	0.560089	0.759244	0.045221	0.54017	0.110852	0.537911	0.934418	0.573776	0.365234	0.250545	0.863869	0.108769	0.0556473	0.690457	0.752701	0.301829	0.0969686	0.996922	0.596123	0.972959	0.329478	0.811853	0.396499	0.956983	0.319399	0.0920556	0.986654	0.175337	0.569408	0.773597	0.134914	0.889006	0.120567	0.477122	0.678824	0.711837	0.0697078	0.120692	0.532224	0.275187	0.801821	0.767578	0.616763	0.556994	⋯

@btime $x[1, :]

  22.814 ns (0 allocations: 0 bytes)

DataFrameRow (1000 columns)

900 columns omitted

Row	x1	x2	x3	x4	x5	x6	x7	x8	x9	x10	x11	x12	x13	x14	x15	x16	x17	x18	x19	x20	x21	x22	x23	x24	x25	x26	x27	x28	x29	x30	x31	x32	x33	x34	x35	x36	x37	x38	x39	x40	x41	x42	x43	x44	x45	x46	x47	x48	x49	x50	x51	x52	x53	x54	x55	x56	x57	x58	x59	x60	x61	x62	x63	x64	x65	x66	x67	x68	x69	x70	x71	x72	x73	x74	x75	x76	x77	x78	x79	x80	x81	x82	x83	x84	x85	x86	x87	x88	x89	x90	x91	x92	x93	x94	x95	x96	x97	x98	x99	x100	⋯
	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	⋯
1	0.855908	0.999016	0.790408	0.268832	0.495286	0.749496	0.333178	0.0277931	0.352712	0.329304	0.691565	0.069195	0.796135	0.952	0.73932	0.328348	0.734515	0.126988	0.341958	0.935508	0.282837	0.877078	0.84413	0.870527	0.502392	0.0219476	0.73117	0.242422	0.0977879	0.730115	0.55251	0.654095	0.640858	0.838408	0.510724	0.0346715	0.136903	0.249659	0.273487	0.232657	0.957472	0.844805	0.812701	0.589498	0.0508585	0.688785	0.969166	0.847723	0.681458	0.36317	0.246428	0.875776	0.18572	0.200951	0.684077	0.727957	0.560089	0.759244	0.045221	0.54017	0.110852	0.537911	0.934418	0.573776	0.365234	0.250545	0.863869	0.108769	0.0556473	0.690457	0.752701	0.301829	0.0969686	0.996922	0.596123	0.972959	0.329478	0.811853	0.396499	0.956983	0.319399	0.0920556	0.986654	0.175337	0.569408	0.773597	0.134914	0.889006	0.120567	0.477122	0.678824	0.711837	0.0697078	0.120692	0.532224	0.275187	0.801821	0.767578	0.616763	0.556994	⋯

@btime view($x, 1:1, :)

  22.823 ns (0 allocations: 0 bytes)

1×1000 SubDataFrame

900 columns omitted

Row	x1	x2	x3	x4	x5	x6	x7	x8	x9	x10	x11	x12	x13	x14	x15	x16	x17	x18	x19	x20	x21	x22	x23	x24	x25	x26	x27	x28	x29	x30	x31	x32	x33	x34	x35	x36	x37	x38	x39	x40	x41	x42	x43	x44	x45	x46	x47	x48	x49	x50	x51	x52	x53	x54	x55	x56	x57	x58	x59	x60	x61	x62	x63	x64	x65	x66	x67	x68	x69	x70	x71	x72	x73	x74	x75	x76	x77	x78	x79	x80	x81	x82	x83	x84	x85	x86	x87	x88	x89	x90	x91	x92	x93	x94	x95	x96	x97	x98	x99	x100	⋯
	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	⋯
1	0.855908	0.999016	0.790408	0.268832	0.495286	0.749496	0.333178	0.0277931	0.352712	0.329304	0.691565	0.069195	0.796135	0.952	0.73932	0.328348	0.734515	0.126988	0.341958	0.935508	0.282837	0.877078	0.84413	0.870527	0.502392	0.0219476	0.73117	0.242422	0.0977879	0.730115	0.55251	0.654095	0.640858	0.838408	0.510724	0.0346715	0.136903	0.249659	0.273487	0.232657	0.957472	0.844805	0.812701	0.589498	0.0508585	0.688785	0.969166	0.847723	0.681458	0.36317	0.246428	0.875776	0.18572	0.200951	0.684077	0.727957	0.560089	0.759244	0.045221	0.54017	0.110852	0.537911	0.934418	0.573776	0.365234	0.250545	0.863869	0.108769	0.0556473	0.690457	0.752701	0.301829	0.0969686	0.996922	0.596123	0.972959	0.329478	0.811853	0.396499	0.956983	0.319399	0.0920556	0.986654	0.175337	0.569408	0.773597	0.134914	0.889006	0.120567	0.477122	0.678824	0.711837	0.0697078	0.120692	0.532224	0.275187	0.801821	0.767578	0.616763	0.556994	⋯

@btime $x[1:1, 1:20]

  4.098 μs (51 allocations: 4.28 KiB)

1×20 DataFrame

Row	x1	x2	x3	x4	x5	x6	x7	x8	x9	x10	x11	x12	x13	x14	x15	x16	x17	x18	x19	x20
	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64
1	0.855908	0.999016	0.790408	0.268832	0.495286	0.749496	0.333178	0.0277931	0.352712	0.329304	0.691565	0.069195	0.796135	0.952	0.73932	0.328348	0.734515	0.126988	0.341958	0.935508

@btime $x[1, 1:20]

  23.447 ns (0 allocations: 0 bytes)

DataFrameRow (20 columns)

Row	x1	x2	x3	x4	x5	x6	x7	x8	x9	x10	x11	x12	x13	x14	x15	x16	x17	x18	x19	x20
	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64
1	0.855908	0.999016	0.790408	0.268832	0.495286	0.749496	0.333178	0.0277931	0.352712	0.329304	0.691565	0.069195	0.796135	0.952	0.73932	0.328348	0.734515	0.126988	0.341958	0.935508

@btime view($x, 1:1, 1:20)

  23.156 ns (0 allocations: 0 bytes)

1×20 SubDataFrame

Row	x1	x2	x3	x4	x5	x6	x7	x8	x9	x10	x11	x12	x13	x14	x15	x16	x17	x18	x19	x20
	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64
1	0.855908	0.999016	0.790408	0.268832	0.495286	0.749496	0.333178	0.0277931	0.352712	0.329304	0.691565	0.069195	0.796135	0.952	0.73932	0.328348	0.734515	0.126988	0.341958	0.935508

Performance tips

Contents

Performance tips#

Access by column number is faster than by name#

When working with data DataFrame use barrier functions or type annotation#

Consider using delayed DataFrame creation technique#

You can add rows to a DataFrame in place and it is fast#

Allowing missing as well as categorical slows down computations#

When aggregating use column selector and prefer integer, categorical, or pooled array grouping variable#

Use views instead of materializing a new DataFrame#

Consider using delayed `DataFrame` creation technique#