Basic information about a data frame

using DataFrames

Let’s start by creating a DataFrame object, x, so that we can learn how to get information on that data frame.

x = DataFrame(A=[1, 2], B=[1.0, missing], C=["a", "b"])

2×3 DataFrame

Row	A	B	C
	Int64	Float64?	String
1	1	1.0	a
2	2	missing	b

The standard size function works to get dimensions of the DataFrame,

size(x), size(x, 1), size(x, 2)

((2, 3), 2, 3)

as well as nrow and ncol from R.

nrow(x), ncol(x)

(2, 3)

describe gives basic summary statistics of data in your DataFrame (check out the help of describe for information on how to customize shown statistics).

describe(x)

3×7 DataFrame

Row	variable	mean	min	median	max	nmissing	eltype
	Symbol	Union…	Any	Union…	Any	Int64	Type
1	A	1.5	1	1.5	2	0	Int64
2	B	1.0	1.0	1.0	1.0	1	Union{Missing, Float64}
3	C		a		b	0	String

you can limit the columns shown by describe using cols keyword argument

describe(x, cols=1:2)

2×7 DataFrame

Row	variable	mean	min	median	max	nmissing	eltype
	Symbol	Float64	Real	Float64	Real	Int64	Type
1	A	1.5	1	1.5	2	0	Int64
2	B	1.0	1.0	1.0	1.0	1	Union{Missing, Float64}

names will return the names of all columns as strings

names(x)

3-element Vector{String}:
 "A"
 "B"
 "C"

you can also get column names with a given element type (eltype):

names(x, String)

1-element Vector{String}:
 "C"

use propertynames to get a vector of Symbols:

propertynames(x)

3-element Vector{Symbol}:
 :A
 :B
 :C

eltype on eachcol(x) returns element types of columns:

eltype.(eachcol(x))

3-element Vector{Type}:
 Int64
 Union{Missing, Float64}
 String

Here we create some large DataFrame

y = DataFrame(rand(1:10, 1000, 10), :auto)

1000×10 DataFrame

975 rows omitted

Row	x1	x2	x3	x4	x5	x6	x7	x8	x9	x10
	Int64	Int64	Int64	Int64	Int64	Int64	Int64	Int64	Int64	Int64
1	1	3	6	5	3	3	1	8	8	3
2	10	8	1	7	8	3	10	10	7	9
3	9	9	9	4	2	8	3	4	10	8
4	6	4	6	7	8	2	3	5	8	1
5	3	1	8	5	4	6	4	8	5	1
6	7	2	1	7	3	8	6	8	2	4
7	2	9	9	4	6	7	8	7	2	1
8	3	3	5	10	7	7	8	2	2	7
9	8	10	6	2	3	7	3	2	1	3
10	3	9	9	5	5	9	5	6	3	6
11	6	3	7	3	4	5	7	8	9	7
12	9	3	5	1	8	1	3	2	6	8
13	7	1	7	9	3	4	10	1	5	1
⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮
989	3	10	9	8	3	7	9	6	3	9
990	8	10	1	8	3	10	4	1	2	5
991	8	3	3	1	6	8	4	10	7	9
992	6	5	1	8	3	7	7	6	6	10
993	3	7	10	7	3	10	2	6	1	4
994	2	9	2	9	3	2	8	3	1	1
995	2	8	4	3	10	10	3	7	5	2
996	7	3	1	7	5	5	2	2	3	2
997	7	4	7	2	2	6	8	4	6	8
998	4	8	1	1	3	3	2	3	8	8
999	1	9	4	4	10	6	4	3	8	2
1000	7	1	4	2	1	8	1	1	7	3

and then we can use first to peek into its first few rows

first(y, 5)

5×10 DataFrame

Row	x1	x2	x3	x4	x5	x6	x7	x8	x9	x10
	Int64	Int64	Int64	Int64	Int64	Int64	Int64	Int64	Int64	Int64
1	1	3	6	5	3	3	1	8	8	3
2	10	8	1	7	8	3	10	10	7	9
3	9	9	9	4	2	8	3	4	10	8
4	6	4	6	7	8	2	3	5	8	1
5	3	1	8	5	4	6	4	8	5	1

and last to see its bottom rows.

last(y, 3)

3×10 DataFrame

Row	x1	x2	x3	x4	x5	x6	x7	x8	x9	x10
	Int64	Int64	Int64	Int64	Int64	Int64	Int64	Int64	Int64	Int64
1	4	8	1	1	3	3	2	3	8	8
2	1	9	4	4	10	6	4	3	8	2
3	7	1	4	2	1	8	1	1	7	3

Using first and last without number of rows will return a first/last DataFrameRow in the DataFrame

first(y)

DataFrameRow (10 columns)

Row	x1	x2	x3	x4	x5	x6	x7	x8	x9	x10
	Int64	Int64	Int64	Int64	Int64	Int64	Int64	Int64	Int64	Int64
1	1	3	6	5	3	3	1	8	8	3

last(y)

DataFrameRow (10 columns)

Row	x1	x2	x3	x4	x5	x6	x7	x8	x9	x10
	Int64	Int64	Int64	Int64	Int64	Int64	Int64	Int64	Int64	Int64
1000	7	1	4	2	1	8	1	1	7	3

Displaying large data frames

Create a wide and tall data frame:

df = DataFrame(rand(100, 100), :auto)

100×100 DataFrame

75 rows omitted

Row	x1	x2	x3	x4	x5	x6	x7	x8	x9	x10	x11	x12	x13	x14	x15	x16	x17	x18	x19	x20	x21	x22	x23	x24	x25	x26	x27	x28	x29	x30	x31	x32	x33	x34	x35	x36	x37	x38	x39	x40	x41	x42	x43	x44	x45	x46	x47	x48	x49	x50	x51	x52	x53	x54	x55	x56	x57	x58	x59	x60	x61	x62	x63	x64	x65	x66	x67	x68	x69	x70	x71	x72	x73	x74	x75	x76	x77	x78	x79	x80	x81	x82	x83	x84	x85	x86	x87	x88	x89	x90	x91	x92	x93	x94	x95	x96	x97	x98	x99	x100
	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64	Float64
1	0.789962	0.0624244	0.735969	0.460358	0.995564	0.778658	0.653409	0.8649	0.744414	0.718615	0.817303	0.842155	0.90281	0.678668	0.196716	0.400046	0.975851	0.408243	0.129669	0.1608	0.728455	0.810081	0.94314	0.949985	0.382518	0.398125	0.316978	0.565363	0.217461	0.181603	0.170618	0.994069	0.686484	0.0137787	0.0881711	0.637445	0.59441	0.310437	0.087433	0.980031	0.176949	0.0105923	0.367188	0.96485	0.50751	0.144118	0.635031	0.51495	0.955666	0.196045	0.93247	0.348846	0.151913	0.511562	0.371784	0.276612	0.927345	0.574142	0.256917	0.823203	0.80771	0.34148	0.520293	0.121687	0.840483	0.204535	0.216077	0.29907	0.739836	0.532872	0.890434	0.662174	0.364033	0.00789043	0.316984	0.891117	0.332019	0.329171	0.25549	0.891567	0.569998	0.565552	0.962039	0.715544	0.819078	0.731385	0.496026	0.6974	0.821201	0.708521	0.525228	0.491904	0.582272	0.91102	0.898055	0.81731	0.948104	0.0559782	0.65191	0.782342
2	0.906916	0.801493	0.916937	0.303229	0.0785325	0.504828	0.11424	0.0990922	0.173723	0.764554	0.661564	0.677853	0.850174	0.426791	0.803742	0.849076	0.629116	0.899457	0.0307474	0.367295	0.276108	0.317422	0.287991	0.733444	0.602123	0.0883216	0.988887	0.0222579	0.421692	0.046251	0.338589	0.658156	0.420062	0.483967	0.269088	0.912539	0.521364	0.398309	0.904374	0.128759	0.845563	0.265137	0.940024	0.909791	0.174972	0.379679	0.72579	0.511081	0.845378	0.768309	0.865279	0.598697	0.961984	0.428272	0.697565	0.475775	0.0471513	0.337792	0.696306	0.128999	0.291203	0.266301	0.25725	0.813233	0.322388	0.765776	0.632218	0.897858	0.536752	0.466092	0.975	0.019028	0.104923	0.0942328	0.253896	0.216972	0.16876	0.199435	0.365484	0.520608	0.580947	0.346234	0.46632	0.387265	0.573617	0.253971	0.935074	0.123601	0.603316	0.2704	0.623716	0.40031	0.336837	0.994516	0.711772	0.498722	0.588764	0.405056	0.762578	0.810863
3	0.191172	0.640318	0.591957	0.768282	0.976985	0.947779	0.0449646	0.702167	0.806419	0.760546	0.462788	0.665196	0.68471	0.333927	0.273586	0.343196	0.614686	0.23853	0.665337	0.640259	0.12159	0.543603	0.746536	0.223446	0.780712	0.139613	0.857278	0.527547	0.303426	0.668916	0.746008	0.852587	0.506523	0.565938	0.736907	0.434405	0.597016	0.817368	0.448656	0.802766	0.578649	0.661492	0.762909	0.886029	0.99614	0.525082	0.932952	0.0848876	0.542636	0.0237858	0.731222	0.403012	0.0551489	0.24299	0.517912	0.850367	0.594318	0.62938	0.0900461	0.281503	0.601483	0.377034	0.781816	0.23537	0.770827	0.301995	0.918257	0.415067	0.88697	0.0819572	0.254009	0.247897	0.902931	0.403027	0.89484	0.0262883	0.0191619	0.0845353	0.69958	0.848151	0.826964	0.0973835	0.521548	0.535922	0.895431	0.228704	0.677998	0.481316	0.0656549	0.412536	0.429962	0.0591372	0.465167	0.189953	0.976226	0.350392	0.535618	0.819281	0.354681	0.596328
4	0.152854	0.519964	0.588042	0.586559	0.355327	0.272534	0.705556	0.94091	0.279872	0.137491	0.926185	0.591034	0.484718	0.427958	0.451731	0.590535	0.481584	0.0655017	0.79735	0.255349	0.60009	0.898634	0.99412	0.410166	0.87283	0.0498867	0.773406	0.646541	0.0560735	0.530586	0.217024	0.337691	0.092964	0.13451	0.127027	0.732276	0.744345	0.0786001	0.236577	0.193732	0.806846	0.0803981	0.167407	0.627898	0.47613	0.618222	0.102311	0.927473	0.91135	0.255356	0.518048	0.486895	0.53087	0.401906	0.689675	0.171761	0.669048	0.0621531	0.59158	0.549113	0.692023	0.481487	0.294933	0.589698	0.0226371	0.764085	0.777256	0.258074	0.87056	0.365023	0.269927	0.672063	0.0639364	0.227701	0.786819	0.811428	0.790961	0.116625	0.454946	0.779165	0.509907	0.958328	0.00392277	0.739633	0.100159	0.752988	0.150669	0.0670422	0.509912	0.747937	0.781694	0.179175	0.877489	0.266761	0.750775	0.604885	0.227372	0.102778	0.519049	0.83224
5	0.438492	0.979425	0.0891575	0.945882	0.730485	0.642267	0.878645	0.163101	0.0735955	0.21277	0.188028	0.43106	0.773998	0.391424	0.274476	0.952568	0.319698	0.25497	0.890972	0.370678	0.946526	0.796653	0.0266869	0.28131	0.558991	0.151908	0.017412	0.605246	0.681085	0.931187	0.339746	0.99863	0.634039	0.267856	0.628497	0.96877	0.494759	0.10337	0.849618	0.847402	0.173661	0.70678	0.336419	0.674687	0.328679	0.610705	0.0762114	0.116168	0.879191	0.874264	0.365542	0.705772	0.995417	0.639454	0.718854	0.120333	0.931438	0.204427	0.0598947	0.393186	0.563545	0.657745	0.501017	0.416358	0.897751	0.563896	0.88921	0.705617	0.102253	0.624221	0.791442	0.858432	0.342829	0.610995	0.263944	0.813046	0.682503	0.311277	0.33614	0.373246	0.380557	0.842904	0.0340886	0.977756	0.827696	0.905881	0.633318	0.201186	0.0448009	0.136333	0.852752	0.890638	0.207635	0.593463	0.922308	0.302028	0.74313	0.911329	0.741833	0.189258
6	0.978017	0.685007	0.13794	0.527277	0.959323	0.353505	0.883649	0.497946	0.508299	0.364806	0.884226	0.318954	0.191665	0.529934	0.283208	0.584212	0.642504	0.397067	0.439616	0.332097	0.758513	0.767744	0.252775	0.874276	0.512381	0.855634	0.768054	0.247361	0.893618	0.677997	0.849732	0.200949	0.916447	0.473907	0.887693	0.204561	0.3219	0.0677458	0.403429	0.210113	0.648227	0.840452	0.740517	0.821338	0.248555	0.248092	0.00993002	0.950479	0.508806	0.668246	0.435753	0.543561	0.51706	0.0568403	0.573082	0.772492	0.58292	0.657442	0.580815	0.0526001	0.937641	0.274714	0.157932	0.118331	0.483643	0.862516	0.896659	0.374468	0.49819	0.337619	0.360727	0.35471	0.0689635	0.86436	0.825771	0.859273	0.787955	0.937837	0.596613	0.175667	0.957355	0.812427	0.125764	0.918091	0.4757	0.864551	0.192622	0.874821	0.491157	0.830136	0.702021	0.580504	0.63501	0.997995	0.113292	0.355345	0.448688	0.31359	0.742707	0.784953
7	0.355382	0.387504	0.0386789	0.817752	0.920758	0.947085	0.267968	0.508267	0.0744801	0.0416993	0.416237	0.154995	0.795377	0.863224	0.725302	0.858817	0.556121	0.0196886	0.640486	0.664734	0.801778	0.174084	0.747256	0.47967	0.0561347	0.947339	0.969936	0.392161	0.441867	0.962798	0.432254	0.211471	0.503355	0.233193	0.279647	0.907691	0.802452	0.538268	0.238805	0.473371	0.143532	0.502939	0.878033	0.0284768	0.154575	0.475727	0.980766	0.837161	0.485342	0.355029	0.310826	0.195347	0.00953382	0.513923	0.0947596	0.0375894	0.351014	0.094923	0.513024	0.859913	0.90293	0.789147	0.37969	0.897667	0.0389584	0.189168	0.603927	0.423735	0.839033	0.991428	0.854764	0.43871	0.918347	0.927122	0.489236	0.763886	0.892663	0.19698	0.520183	0.509847	0.706177	0.766694	0.0136834	0.156429	0.132012	0.791273	0.47403	0.274121	0.521274	0.592208	0.698895	0.802799	0.648142	0.0471296	0.633471	0.995833	0.190746	0.431809	0.814291	0.260273
8	0.623965	0.573277	0.380789	0.272908	0.484304	0.343541	0.634027	0.307012	0.862936	0.230815	0.652491	0.79243	0.514264	0.571855	0.0450283	0.741723	0.425562	0.760773	0.848076	0.491073	0.0404891	0.636267	0.00637674	0.243197	0.804827	0.78336	0.301242	0.335687	0.931932	0.247274	0.912292	0.449109	0.795803	0.641221	0.176784	0.969964	0.670471	0.995402	0.0421617	0.522538	0.333508	0.0797808	0.546625	0.215847	0.6817	0.535863	0.260338	0.843419	0.889026	0.506287	0.713106	0.695158	0.642464	0.0573208	0.805722	0.536209	0.624832	0.698036	0.0400037	0.721292	0.960602	0.607294	0.772479	0.738393	0.198255	0.100064	0.802141	0.304446	0.425035	0.905866	0.157392	0.0451183	0.953696	0.752476	0.456385	0.543769	0.867963	0.976257	0.866777	0.0794833	0.959075	0.691634	0.744851	0.421241	0.978213	0.471006	0.489963	0.966642	0.854582	0.769771	0.248433	0.962077	0.0555362	0.628293	0.938299	0.920196	0.140157	0.796787	0.567865	0.407918
9	0.924967	0.774597	0.232752	0.204111	0.366567	0.902385	0.921746	0.473791	0.472789	0.461728	0.511422	0.22958	0.182011	0.0234031	0.741959	0.609675	0.406637	0.398294	0.622153	0.404439	0.437776	0.00542105	0.748615	0.259586	0.524832	0.648169	0.55465	0.48709	0.523374	0.231486	0.592701	0.681264	0.488907	0.430831	0.627875	0.351296	0.220672	0.496731	0.988125	0.915999	0.604512	0.374545	0.403834	0.807666	0.475156	0.359549	0.691342	0.82921	0.842037	0.665612	0.752709	0.326745	0.369885	0.745714	0.501246	0.124589	0.889516	0.866604	0.856362	0.667124	0.446837	0.68607	0.762944	0.705123	0.157266	0.994174	0.16359	0.63707	0.443053	0.764881	0.51556	0.846679	0.00181272	0.200586	0.781975	0.223445	0.838355	0.171095	0.475408	0.0767908	0.86955	0.518845	0.461329	0.795333	0.891451	0.425662	0.995346	0.562735	0.187163	0.751779	0.245888	0.239954	0.570511	0.593536	0.580887	0.93795	0.797648	0.101556	0.130643	0.132767
10	0.870018	0.523782	0.146663	0.0688282	0.910187	0.030617	0.7195	0.424132	0.739784	0.106837	0.913481	0.310718	0.0253776	0.730706	0.69732	0.220479	0.121729	0.772284	0.792031	0.175478	0.242331	0.94515	0.620682	0.25098	0.773386	0.474092	0.64773	0.531523	0.63011	0.47455	0.0115982	0.233914	0.730183	0.240237	0.432014	0.0660656	0.579257	0.801662	0.734037	0.280745	0.467608	0.227433	0.318556	0.070222	0.0707096	0.455186	0.514461	0.314014	0.64161	0.0681791	0.45958	0.483418	0.308741	0.124927	0.880256	0.850842	0.0393761	0.0398096	0.122609	0.488181	0.160364	0.112608	0.220929	0.207202	0.494636	0.525268	0.0256017	0.857939	0.509653	0.624037	0.506282	0.385628	0.363783	0.589266	0.752108	0.80989	0.731094	0.870923	0.663211	0.787562	0.128138	0.826731	0.838876	0.762065	0.694518	0.0854203	0.149463	0.711009	0.895898	0.891102	0.0596153	0.934214	0.405129	0.95902	0.0679759	0.223175	0.731698	0.958341	0.490405	0.0775291
11	0.362548	0.599407	0.0947538	0.739589	0.542192	0.841855	0.0601135	0.194039	0.470178	0.527984	0.527213	0.946075	0.95859	0.51009	0.55743	0.753128	0.267601	0.0490926	0.23975	0.808986	0.671142	0.96051	0.764447	0.496478	0.640831	0.862786	0.130849	0.731722	0.581203	0.424323	0.0735296	0.566577	0.722778	0.815274	0.154186	0.477042	0.311353	0.658095	0.954405	0.56225	0.337264	0.81479	0.176337	0.553602	0.292243	0.811956	0.945509	0.783088	0.248424	0.838007	0.277383	0.505151	0.906155	0.755413	0.430479	0.782704	0.962104	0.56388	0.95343	0.350109	0.64119	0.459333	0.799797	0.308424	0.95662	0.499096	0.907639	0.847775	0.508379	0.156982	0.549768	0.996267	0.329579	0.233731	0.631585	0.688159	0.052376	0.147506	0.152467	0.962346	0.945086	0.448964	0.112802	0.113403	0.684567	0.718678	0.652964	0.246463	0.905186	0.848404	0.500186	0.280353	0.714817	0.947439	0.61223	0.256481	0.886191	0.912358	0.361513	0.211067
12	0.615616	0.848533	0.683536	0.753884	0.105334	0.491558	0.0541334	0.36363	0.237825	0.0794509	0.323071	0.801587	0.553979	0.0928562	0.918386	0.078732	0.278998	0.685108	0.779597	0.0217248	0.173892	0.300055	0.0487842	0.633044	0.654244	0.181167	0.567375	0.854619	0.0697445	0.256387	0.185591	0.919599	0.934497	0.390927	0.738818	0.925873	0.982855	0.170505	0.616483	0.0586832	0.309217	0.0302802	0.856725	0.351917	0.0361207	0.48256	0.573614	0.336081	0.156247	0.141644	0.749402	0.420174	0.114279	0.346357	0.118401	0.716484	0.872654	0.333551	0.899951	0.469164	0.953272	0.577793	0.507059	0.757946	0.0130374	0.485062	0.456176	0.205114	0.939635	0.912074	0.397316	0.415366	0.055941	0.255077	0.0619957	0.1558	0.856876	0.762088	0.455591	0.758976	0.274573	0.696724	0.0307146	0.362178	0.247994	0.522491	0.693922	0.640807	0.24977	0.017836	0.336338	0.99548	0.735818	0.124317	0.41132	0.714684	0.448359	0.299657	0.362069	0.958475
13	0.462013	0.906368	0.782922	0.0450027	0.216445	0.531721	0.199916	0.316595	0.625991	0.112216	0.693513	0.971785	0.885556	0.848221	0.524184	0.156683	0.151868	0.301741	0.644763	0.256146	0.869783	0.0261141	0.345316	0.18832	0.0684432	0.392228	0.0826762	0.172071	0.141608	0.968717	0.939086	0.722096	0.455368	0.461017	0.463184	0.0203132	0.341906	0.434247	0.286317	0.481753	0.885127	0.102202	0.136855	0.3366	0.236358	0.385715	0.128861	0.429646	0.414888	0.820758	0.126101	0.947022	0.406631	0.322354	0.463839	0.554269	0.986428	0.78869	0.464203	0.958466	0.689087	0.760268	0.294495	0.973128	0.604884	0.831911	0.62057	0.198623	0.370587	0.241517	0.565628	0.682043	0.593482	0.748381	0.523238	0.403252	0.686063	0.599615	0.936859	0.496968	0.356396	0.0414967	0.0520849	0.718152	0.599364	0.180371	0.930551	0.912203	0.565355	0.977152	0.825388	0.904074	0.955182	0.0155227	0.728541	0.63332	0.650479	0.717457	0.631643	0.374587
⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮
89	0.280769	0.385492	0.0585452	0.0178141	0.3181	0.17023	0.501149	0.917731	0.603734	0.0955169	0.0973232	0.944897	0.00132417	0.580976	0.165529	0.039724	0.888133	0.273872	0.172742	0.950642	0.251519	0.294921	0.74549	0.991615	0.680301	0.990152	0.200681	0.26473	0.0792035	0.373631	0.579267	0.459689	0.039033	0.587437	0.929283	0.397582	0.0419838	0.632037	0.939106	0.491073	0.625461	0.526602	0.363351	0.0842547	0.578567	0.778179	0.633321	0.001486	0.90662	0.107684	0.222563	0.547652	0.382166	0.241437	0.402704	0.916321	0.128908	0.712597	0.607795	0.961437	0.445001	0.785382	0.677261	0.348534	0.817878	0.542834	0.834988	0.476018	0.821724	0.620973	0.115776	0.882758	0.252971	0.946628	0.677736	0.202076	0.354347	0.882999	0.775248	0.958036	0.668448	0.0131915	0.44857	0.397489	0.421281	0.33918	0.434042	0.619287	0.524504	0.354485	0.802962	0.384289	0.567239	0.389545	0.178408	0.539163	0.389794	0.756947	0.234594	0.00260606
90	0.15863	0.892324	0.195818	0.189546	0.12159	0.462442	0.0446866	0.661574	0.114463	0.235853	0.190155	0.000373413	0.00368981	0.954947	0.244063	0.101028	0.0818196	0.717605	0.639297	0.405519	0.0261208	0.137022	0.922165	0.293154	0.836947	0.49416	0.304998	0.553104	0.782369	0.609711	0.566448	0.953891	0.263249	0.572713	0.866514	0.630133	0.861353	0.0946774	0.959546	0.734624	0.931596	0.15181	0.221525	0.977204	0.675744	0.915331	0.855707	0.946596	0.433012	0.945452	0.111026	0.183187	0.734269	0.3462	0.906348	0.984058	0.970597	0.404078	0.824375	0.885151	0.763847	0.875424	0.910119	0.384915	0.98971	0.11249	0.149937	0.975611	0.207411	0.234994	0.520353	0.996016	0.740065	0.274514	0.0558998	0.728554	0.223162	0.913611	0.307939	0.525881	0.147426	0.177258	0.905207	0.888296	0.751841	0.487407	0.198813	0.785166	0.0951316	0.571764	0.422065	0.467302	0.597503	0.43933	0.874329	0.451494	0.851046	0.116624	0.301025	0.684658
91	0.761741	0.227345	0.515141	0.733511	0.421822	0.78358	0.43322	0.226086	0.058077	0.799261	0.581793	0.111209	0.134759	0.552973	0.158635	0.931679	0.789016	0.905897	0.337439	0.0529721	0.128442	0.787425	0.715367	0.456998	0.804064	0.450039	0.845572	0.419271	0.506906	0.861448	0.263005	0.607297	0.226538	0.215837	0.504686	0.500379	0.321679	0.81371	0.112133	0.675506	0.0118633	0.632589	0.966534	0.335967	0.886913	0.952092	0.416806	0.131716	0.36855	0.420991	0.0238001	0.302086	0.21468	0.294167	0.0375048	0.308672	0.795785	0.574172	0.798863	0.363535	0.616006	0.927874	0.4421	0.7504	0.209376	0.323075	0.941857	0.824667	0.893805	0.163432	0.197442	0.279206	0.130657	0.194932	0.416209	0.603471	0.84675	0.485001	0.514717	0.405339	0.383213	0.552306	0.904849	0.0615813	0.444421	0.296819	0.202913	0.882236	0.746749	0.574821	0.191713	0.295837	0.476607	0.141302	0.158083	0.141696	0.969505	0.791368	0.705662	0.038993
92	0.225268	0.239495	0.422271	0.95938	0.425413	0.542066	0.0346159	0.242841	0.153493	0.77317	0.534448	0.987856	0.263189	0.27882	0.755924	0.989297	0.285997	0.352256	0.287437	0.320734	0.0338925	0.114634	0.581964	0.916226	0.69835	0.442443	0.917124	0.0663897	0.578314	0.653805	0.806074	0.00593384	0.602821	0.395626	0.00426104	0.618234	0.50982	0.167969	0.966683	0.584823	0.833108	0.972182	0.960039	0.845157	0.316593	0.931999	0.357477	0.0160759	0.683118	0.347728	0.143008	0.0524016	0.220243	0.557718	0.249238	0.886247	0.0876338	0.362077	0.804643	0.578342	0.96316	0.399454	0.384779	0.929979	0.406629	0.645757	0.673865	0.0822934	0.739298	0.983528	0.368873	0.504532	0.488396	0.661342	0.148015	0.00542962	0.0279379	0.448618	0.749486	0.13582	0.152731	0.32536	0.982172	0.632062	0.0651492	0.651519	0.104345	0.963408	0.955775	0.0513819	0.608345	0.262896	0.684068	0.394801	0.502742	0.821472	0.0774761	0.108518	0.982675	0.906776
93	0.717629	0.140878	0.747755	0.595891	0.27867	0.374846	0.88069	0.270184	0.853435	0.477647	0.804555	0.919529	0.404235	0.751565	0.624275	0.87306	0.328693	0.494063	0.403297	0.594706	0.473244	0.727695	0.772057	0.281999	0.994671	0.279026	0.274097	0.53135	0.322491	0.979215	0.993971	0.761147	0.791821	0.0209872	0.38958	0.370931	0.461343	0.602745	0.215164	0.243521	0.330244	0.810435	0.567199	0.0305275	0.316676	0.88219	0.243579	0.784181	0.536952	0.993897	0.0559512	0.37994	0.556133	0.786242	0.710635	0.135289	0.355904	0.349001	0.827598	0.665336	0.976985	0.524632	0.798625	0.899973	0.716689	0.500235	0.416318	0.417066	0.217658	0.810996	0.667468	0.429859	0.977843	0.873982	0.201755	0.774194	0.398476	0.474525	0.364497	0.285219	0.441581	0.880467	0.341518	0.997719	0.075591	0.513803	0.493363	0.159287	0.911703	0.899411	0.630994	0.564332	0.494753	0.293181	0.60395	0.995029	0.814803	0.413212	0.0642455	0.634372
94	0.932557	0.702032	0.219581	0.886079	0.588761	0.271663	0.506945	0.818636	0.0750715	0.805134	0.541555	0.584367	0.913684	0.861835	0.861393	0.454013	0.0882914	0.520239	0.259026	0.42586	0.559467	0.150548	0.781107	0.281346	0.366376	0.337876	0.384217	0.200233	0.867859	0.280687	0.166237	0.404492	0.418316	0.814588	0.0886978	0.43562	0.168099	0.839854	0.531314	0.406109	0.029576	0.496501	0.70359	0.482049	0.353881	0.0192666	0.800116	0.166103	0.649093	0.244153	0.60502	0.641778	0.965286	0.0540275	0.959171	0.825505	0.373089	0.817307	0.116926	0.454165	0.365613	0.265781	0.00420759	0.330985	0.854153	0.674102	0.487941	0.972557	0.538995	0.706153	0.759531	0.980115	0.920845	0.0146197	0.569798	0.569162	0.0938785	0.823528	0.609963	0.699218	0.552689	0.501404	0.510127	0.0125349	0.690941	0.634956	0.950028	0.811396	0.0932884	0.454412	0.624139	0.812365	0.379762	0.0730752	0.647844	0.505606	0.218623	0.279447	0.601069	0.41675
95	0.957338	0.00108312	0.0452597	0.0415551	0.0603021	0.932539	0.162327	0.554332	0.00626094	0.411086	0.905709	0.147108	0.588769	0.754456	0.0549182	0.683031	0.240543	0.917737	0.596222	0.142523	0.208593	0.0570086	0.190565	0.966917	0.45958	0.51624	0.474936	0.327285	0.764723	0.705456	0.2352	0.848233	0.426502	0.00218772	0.544734	0.176609	0.394878	0.533875	0.129297	0.145668	0.819916	0.613221	0.0732765	0.0298999	0.0145071	0.492178	0.644573	0.289402	0.0285995	0.521927	0.471034	0.896232	0.436583	0.181619	0.906494	0.129531	0.609699	0.742403	0.975199	0.161736	0.749093	0.474684	0.476512	0.995447	0.321733	0.464786	0.224941	0.162295	0.781067	0.909128	0.294359	0.732696	0.387156	0.477805	0.617713	0.142863	0.0948252	0.838613	0.428479	0.850149	0.360135	0.579448	0.783899	0.804317	0.734718	0.131184	0.16023	0.716123	0.872733	0.521906	0.966209	0.460746	0.571613	0.873984	0.123332	0.674525	0.752319	0.915687	0.337797	0.733894
96	0.217519	0.0452365	0.96423	0.987456	0.199681	0.927534	0.878267	0.378207	0.872008	0.238098	0.257614	0.633971	0.976785	0.663263	0.390687	0.353285	0.613882	0.741835	0.107931	0.159643	0.956933	0.504997	0.2803	0.433169	0.0156274	0.879004	0.726108	0.683416	0.925806	0.593891	0.454665	0.132348	0.366213	0.0234535	0.729959	0.557051	0.221222	0.568313	0.0348786	0.312595	0.989744	0.0805006	0.824732	0.0477457	0.93347	0.428167	0.326458	0.432381	0.140956	0.581844	0.183974	0.816084	0.390274	0.813328	0.701694	0.838039	0.345457	0.947728	0.329988	0.87144	0.717956	0.403999	0.200429	0.206937	0.590591	0.631804	0.341831	0.720018	0.22841	0.555394	0.236254	0.599366	0.166311	0.635677	0.935486	0.364833	0.924616	0.948936	0.478977	0.951591	0.0633799	0.10812	0.954649	0.220105	0.58744	0.978821	0.212994	0.816767	0.852291	0.44156	0.639763	0.0361241	0.980516	0.139843	0.999062	0.637438	0.218261	0.303185	0.874953	0.308514
97	0.775986	0.767202	0.244489	0.912083	0.464894	0.428593	0.233654	0.270441	0.662784	0.685429	0.913007	0.419868	0.0447673	0.100636	0.530166	0.759601	0.105696	0.878249	0.812319	0.133492	0.123747	0.627862	0.803189	0.176159	0.893605	0.509833	0.89563	0.838178	0.824733	0.21917	0.270275	0.631983	0.480538	0.99744	0.00624713	0.0820833	0.443615	0.164536	0.309013	0.26152	0.0315589	0.266062	0.238658	0.740117	0.420759	0.894284	0.401152	0.406765	0.218936	0.509436	0.476413	0.642263	0.814002	0.496054	0.923606	0.220304	0.771778	0.295205	0.994386	0.760525	0.179646	0.286279	0.00984692	0.106695	0.923622	0.919796	0.719881	0.900047	0.387818	0.794021	0.486981	0.92123	0.0885311	0.667391	0.61822	0.095412	0.784422	0.395569	0.107845	0.281075	0.237278	0.409729	0.669498	0.0539078	0.520631	0.201034	0.476508	0.957075	0.920235	0.376551	0.360892	0.793829	0.572212	0.693187	0.654635	0.916616	0.587588	0.560166	0.200409	0.0910738
98	0.522384	0.740745	0.807622	0.15677	0.842836	0.485635	0.857848	0.922281	0.409467	0.0932816	0.178318	0.732183	0.0223056	0.643492	0.954521	0.916175	0.618734	0.961502	0.214451	0.593554	0.284302	0.652227	0.662953	0.902847	0.740648	0.418843	0.598326	0.320145	0.626925	0.332444	0.94481	0.00951962	0.408272	0.62549	0.727359	0.713845	0.528716	0.0417095	0.953102	0.442172	0.982232	0.356923	0.640211	0.938885	0.279069	0.997778	0.0567664	0.573613	0.239674	0.0670827	0.542654	0.373962	0.750205	0.237124	0.629047	0.374039	0.711036	0.668241	0.685765	0.339248	0.367468	0.461048	0.48342	0.3269	0.717611	0.947521	0.318631	0.0113016	0.200263	0.566034	0.132854	0.22167	0.421606	0.851707	0.411302	0.483742	0.583698	0.755573	0.983859	0.413706	0.969635	0.792034	0.568518	0.226961	0.0210344	0.70362	0.0906433	0.910967	0.560627	0.150792	0.14677	0.66063	0.051029	0.745824	0.613213	0.26847	0.0778211	0.0343551	0.0733916	0.109837
99	0.512551	0.294041	0.213607	0.895076	0.816713	0.363689	0.890063	0.359214	0.772196	0.641827	0.609222	0.709287	0.447994	0.489238	0.320749	0.919799	0.324657	0.660815	0.837899	0.0683506	0.154625	0.0773542	0.184497	0.33372	0.907779	0.52356	0.243873	0.0881471	0.239157	0.193352	0.725013	0.357106	0.737941	0.490811	0.774828	0.447774	0.468067	0.2418	0.939311	0.214387	0.170522	0.116166	0.077245	0.331224	0.808439	0.334034	0.36761	0.96792	0.893116	0.660393	0.744993	0.936395	0.79273	0.409249	0.292189	0.138785	0.25867	0.16999	0.391777	0.906732	0.606501	0.475705	0.557534	0.0124858	0.714136	0.156939	0.87724	0.540045	0.148772	0.10824	0.306888	0.409229	0.756949	0.64836	0.209419	0.677418	0.554342	0.0937178	0.60964	0.127878	0.221488	0.51633	0.103558	0.487593	0.998254	0.495458	0.0962333	0.0497143	0.18801	0.235706	0.254162	0.327916	0.0550599	0.879554	0.422294	0.509737	0.187778	0.270036	0.877707	0.467502
100	0.61419	0.176781	0.0414305	0.791225	0.0696568	0.889425	0.0994257	0.809057	0.53631	0.296434	0.893624	0.65167	0.848323	0.879734	0.859981	0.316221	0.440416	0.384687	0.122039	0.644787	0.262758	0.151728	0.225896	0.724359	0.197822	0.719209	0.0673743	0.298427	0.967721	0.443425	0.332389	0.0993715	0.50076	0.152471	0.0216056	0.454134	0.453329	0.752843	0.223069	0.717538	0.861253	0.0390536	0.38516	0.115852	0.2124	0.819144	0.453541	0.27031	0.384087	0.188818	0.062745	0.564858	0.35968	0.672591	0.831338	0.0351665	0.0429604	0.0122383	0.699613	0.845458	0.931469	0.893151	0.0706975	0.86739	0.275542	0.0334133	0.174198	0.108096	0.692865	0.723792	0.732822	0.188744	0.442734	0.363643	0.609089	0.213475	0.629268	0.939475	0.773375	0.755643	0.6445	0.825181	0.274481	0.0517758	0.433448	0.874557	0.417765	0.867131	0.594601	0.814286	0.797631	0.736357	0.7123	0.938963	0.669559	0.574795	0.415125	0.41307	0.934199	0.179651

we can see that 92 of its columns were not printed. Also we get its first 30 rows. You can easily change this behavior by changing the value of ENV["LINES"] and ENV["COLUMNS"].

withenv("LINES" => 10, "COLUMNS" => 200) do
    show(df)
end

100×100 DataFrame
 Row │ x1        x2         x3         x4        x5         x6        x7         x8         x9        x10        x11       x12       x13        x14       x15       x16       x17       x18        x19 ⋯
     │ Float64   Float64    Float64    Float64   Float64    Float64   Float64    Float64    Float64   Float64    Float64   Float64   Float64    Float64   Float64   Float64   Float64   Float64    Flo ⋯
─────┼──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
   1 │ 0.789962  0.0624244  0.735969   0.460358  0.995564   0.778658  0.653409   0.8649     0.744414  0.718615   0.817303  0.842155  0.90281    0.678668  0.196716  0.400046  0.975851  0.408243   0.1 ⋯
  ⋮  │    ⋮          ⋮          ⋮         ⋮          ⋮         ⋮          ⋮          ⋮         ⋮          ⋮         ⋮         ⋮          ⋮         ⋮         ⋮         ⋮         ⋮          ⋮          ⋱
                                                                                                                                                                          82 columns and 99 rows omitted

Most elementary get and set operations

Given the DataFrame x we have created earlier, here are various ways to grab one of its columns as a Vector.

x = DataFrame(A=[1, 2], B=[1.0, missing], C=["a", "b"])

2×3 DataFrame

Row	A	B	C
	Int64	Float64?	String
1	1	1.0	a
2	2	missing	b

all get the vector stored in our DataFrame without copying it

x.A, x[!, 1], x[!, :A]

([1, 2], [1, 2], [1, 2])

the same using string indexing

x."A", x[!, "A"]

([1, 2], [1, 2])

note that this creates a copy

x[:, 1]

2-element Vector{Int64}:
 1
 2

x[:, 1] === x[:, 1]

false

To grab one row as a DataFrame, we can index as follows.

x[1:1, :]

1×3 DataFrame

Row	A	B	C
	Int64	Float64?	String
1	1	1.0	a

this produces a DataFrameRow which is treated as 1-dimensional object similar to a NamedTuple

x[1, :]

DataFrameRow (3 columns)

Row	A	B	C
	Int64	Float64?	String
1	1	1.0	a

We can grab a single cell or element with the same syntax to grab an element of an array.

x[1, 1]

1

or a new DataFrame that is a subset of rows and columns

x[1:2, 1:2]

2×2 DataFrame

Row	A	B
	Int64	Float64?
1	1	1.0
2	2	missing

You can also use Regex to select columns and Not from InvertedIndices.jl both to select rows and columns

x[Not(1), r"A"]

1×1 DataFrame

Row	A
	Int64
1	2

! indicates that underlying columns are not copied

x[!, Not(1)]

2×2 DataFrame

Row	B	C
	Float64?	String
1	1.0	a
2	missing	b

: means that the columns will get copied

x[:, Not(1)]

2×2 DataFrame

Row	B	C
	Float64?	String
1	1.0	a
2	missing	b

Assignment of a scalar to a data frame can be done in ranges using broadcasting:

x[1:2, 1:2] .= 1
x

2×3 DataFrame

Row	A	B	C
	Int64	Float64?	String
1	1	1.0	a
2	1	1.0	b

Assignment of a vector of length equal to the number of assigned rows using broadcasting

x[1:2, 1:2] .= [1, 2]
x

2×3 DataFrame

Row	A	B	C
	Int64	Float64?	String
1	1	1.0	a
2	2	2.0	b

Assignment or of another data frame of matching size and column names, again using broadcasting:

x[1:2, 1:2] .= DataFrame([5 6; 7 8], [:A, :B])
x

2×3 DataFrame

Row	A	B	C
	Int64	Float64?	String
1	5	6.0	a
2	7	8.0	b

Caution

With df[!, :col] and df.col syntax you get a direct (non copying) access to a column of a data frame. This is potentially unsafe as you can easily corrupt data in the df data frame if you resize, sort, etc. the column obtained in this way. Therefore such access should be used with caution.

Similarly df[!, cols] when cols is a collection of columns produces a new data frame that holds the same (not copied) columns as the source df data frame. Similarly, modifying the data frame obtained via df[!, cols] might cause problems with the consistency of df.

The df[:, :col] and df[:, cols] syntaxes always copy columns so they are safe to use (and should generally be preferred except for performance or memory critical use cases).

Here are examples of how Cols and Between can be used to select columns of a data frame.

x = DataFrame(rand(4, 5), :auto)

4×5 DataFrame

Row	x1	x2	x3	x4	x5
	Float64	Float64	Float64	Float64	Float64
1	0.143664	0.243967	0.541132	0.90432	0.253281
2	0.649823	0.974319	0.523402	0.547555	0.906718
3	0.532381	0.154004	0.618354	0.0428888	0.897352
4	0.878731	0.371432	0.413427	0.829586	0.771033

x[:, Between(:x2, :x4)]

4×3 DataFrame

Row	x2	x3	x4
	Float64	Float64	Float64
1	0.243967	0.541132	0.90432
2	0.974319	0.523402	0.547555
3	0.154004	0.618354	0.0428888
4	0.371432	0.413427	0.829586

x[:, Cols("x1", Between("x2", "x4"))]

4×4 DataFrame

Row	x1	x2	x3	x4
	Float64	Float64	Float64	Float64
1	0.143664	0.243967	0.541132	0.90432
2	0.649823	0.974319	0.523402	0.547555
3	0.532381	0.154004	0.618354	0.0428888
4	0.878731	0.371432	0.413427	0.829586

Views

You can simply create a view of a DataFrame (it is more efficient than creating a materialized selection). Here are the possible return value options.

@view x[1:2, 1]

2-element view(::Vector{Float64}, 1:2) with eltype Float64:
 0.14366370663453387
 0.6498227286696711

@view x[1, 1]

0-dimensional view(::Vector{Float64}, 1) with eltype Float64:
0.14366370663453387

a DataFrameRow, the same as for x[1, 1:2] without a view

@view x[1, 1:2]

DataFrameRow (2 columns)

Row	x1	x2
	Float64	Float64
1	0.143664	0.243967

a SubDataFrame

@view x[1:2, 1:2]

2×2 SubDataFrame

Row	x1	x2
	Float64	Float64
1	0.143664	0.243967
2	0.649823	0.974319

Adding new columns to a data frame

df = DataFrame()

0×0 DataFrame

using setproperty! (element assignment)

x = [1, 2, 3]
df.a = x
df

3×1 DataFrame

Row	a
	Int64
1	1
2	2
3	3

no copy is performed (sharing the same memory address)

df.a === x

true

using setindex!

df[!, :b] = x
df[:, :c] = x
df

3×3 DataFrame

Row	a	b	c
	Int64	Int64	Int64
1	1	1	1
2	2	2	2
3	3	3	3

no copy is performed

df.b === x

true

With copying ! and : has different effects

df.c === x

false

Element-wise assignment

df[!, :d] .= x
df[:, :e] .= x
df

3×5 DataFrame

Row	a	b	c	d	e
	Int64	Int64	Int64	Int64	Int64
1	1	1	1	1	1
2	2	2	2	2	2
3	3	3	3	3	3

both copy, so in this case ! and : has the same effect

df.d === x, df.e === x

(false, false)

note that in our data frame columns :a and :b store the vector x (not a copy)

df.a === df.b === x

true

This can lead to silent errors. For example this code leads to a bug (note that calling pairs on eachcol(df) creates an iterator of (column name, column) pairs):

try
    for (n, c) in pairs(eachcol(df))
        println("$n: ", pop!(c))
    end
catch e
    show(e)
end

a: 3
b: 2
c: 3
d: 3
e: 3

note that for column :b we printed 2 as 3 was removed from it when we used pop! on column :a. Such mistakes sometimes happen. Because of this DataFrames.jl performs consistency checks before doing an expensive operation (most notably before showing a data frame).

try
    show(df)
catch e
    show(e)
end

AssertionError("Data frame is corrupt: length of column :c (2) does not match length of column 1 (1). The column vector has likely been resized unintentionally (either directly or because it is shared with another data frame).")

We can investigate the columns to find out what happened:

collect(pairs(eachcol(df)))

5-element Vector{Pair{Symbol, AbstractVector}}:
 :a => [1]
 :b => [1]
 :c => [1, 2]
 :d => [1, 2]
 :e => [1, 2]

The output confirms that the data frame df got corrupted. DataFrames.jl supports a complete set of getindex, getproperty, setindex!, setproperty!, view, broadcasting, and broadcasting assignment operations. The details are explained here: http://juliadata.github.io/DataFrames.jl/latest/lib/indexing/.

Comparisons

using DataFrames

df = DataFrame(rand(2, 3), :auto)

2×3 DataFrame

Row	x1	x2	x3
	Float64	Float64	Float64
1	0.938257	0.811633	0.362354
2	0.975227	0.753141	0.141313

df2 = copy(df)

2×3 DataFrame

Row	x1	x2	x3
	Float64	Float64	Float64
1	0.938257	0.811633	0.362354
2	0.975227	0.753141	0.141313

compares column names and contents

df == df2

true

create a minimally different data frame and use isapprox for comparison

df3 = df2 .+ eps()

2×3 DataFrame

Row	x1	x2	x3
	Float64	Float64	Float64
1	0.938257	0.811633	0.362354
2	0.975227	0.753141	0.141313

df == df3

false

isapprox(df, df3)

true

isapprox(df, df3, atol=eps() / 2)

false

missings are handled as in Julia Base

df = DataFrame(a=missing)

1×1 DataFrame

Row	a
	Missing
1	missing

Equality test shows missing.

df == df

missing

The same object?

df === df

true

isequal(df, df)

true

---

This notebook was generated using Literate.jl.