Skip to content

Instantly share code, notes, and snippets.

@ax3l

ax3l/YT.ipynb

Forked from anokfireball/YT.ipynb
Created January 4, 2017 14:18
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save ax3l/7966c6a995f3ea262cc175317ea1d00f to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save ax3l/7966c6a995f3ea262cc175317ea1d00f to your computer and use it in GitHub Desktop.
Download ZIP
Display the source blob
Display the rendered blob
Raw
YT.ipynb
{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"# YT\n",
"## analysis and visualization of volumetric openPMD\n",
"\n",
"https://git.io/viZvf"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"# Why bother with yet another tool? A tool designed with astrophysics in mind?\n",
"\n",
"You might want to read this if you\n",
"- need a **rough idea** (i.e. a *gorgeous image*) of PIConGPU output (phase-space, histograms, raycasts, ...)\n",
"- need to do **analysis** on particles or fields (min, max, density, ...)\n",
"- need to do any of the above reasonably **quick** (say, you'd like a nice hot coffe with your results about a whole PIC run)\n",
"- want to spend **no more time than this talk** to get started"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "skip"
}
},
"source": [
"## What is yt?\n",
"\n",
"- python library for visualization and analysis of quantitative volumetric data\n",
"- tries to makes you think about *what you want to do* with your data rather than *how you need to do it* (storage on disk, data-formats, ...)\n",
"- offers a range of *user-friendly*, *powerful* instructions\n",
"- ... and a lot of options to fit individual needs (you can easily re-analyse data you have transformed manually)\n",
"- pararallelizable without hassle\n",
"- supports data in (among others) cartesian, cylindrical, sperical coordinates, unstructured mesh data (```ndarray```s can be read directly)\n",
"- driven and developed by a broad scientific community\n",
"- mainly a command-line tool, made somewhat interactive through ipython/jupyter and experimental GUIs\n",
"- now can read openPMD"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"### Installing yt\n",
"To use yt as described below, you will need to install the latest development version.\n",
"You will be able to use yt's features remotely (on clusternodes) with this installation.\n",
"\n",
"```bash\n",
"#!/bin/bash\n",
"export LC_ALL=\"en_US.UTF-8\"\n",
"export LC_CTYPE=\"en_US.UTF-8\"\n",
"\n",
"source /etc/profile.modules\n",
"\n",
"# You might have to adjust the following modules according to your architecture\n",
"module purge\n",
"module load gcc/4.9.2\n",
"module load python/2.7.10\n",
"module load cuda/7.5\n",
"module load openmpi/1.8.6.kepler.cuda75\n",
"module load hdf5-parallel/1.8.15\n",
"module load mercurial/3.4\n",
"\n",
"# Install required python modules inside a virtual environment (highly recommended, even if you can sudo)\n",
"# ONLY CREATE THIS ENVIRONMENT ONCE\n",
"virtualenv YT_ENV\n",
"\n",
"source ~/YT_ENV/bin/activate\n",
"\n",
"# You only need to install this once\n",
"pip install --upgrade pip\n",
"pip install cython mercurial sympy ipython matplotlib h5py mpi4py\n",
"\n",
"# Should you want a jupyter notebook\n",
"pip install jupyter ipyparallel\n",
"jupyter serverextension enable --py ipyparallel \n",
"jupyter nbextension install --py ipyparallel \n",
"jupyter nbextension enable --py ipyparallel\n",
"ipcluster nbextension enable --user\n",
"\n",
"# Download and install the latest yt development version\n",
"hg clone https://C0nsultant@bitbucket.org/C0nsultant/openpmd\n",
"cd openpmd/\n",
"./clean.sh\n",
"hg pull\n",
"hg update yt --clean\n",
"\n",
"python setup.py develop\n",
"```\n",
"\n",
"#### NOTE\n",
"\n",
"You will have to run python inside the virtual environment created during the installation **every time** you want to use yt. \n",
"\n",
"```bash\n",
"source ~/YT_ENV/bin/activate\n",
"```"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"collapsed": false,
"slideshow": {
"slide_type": "skip"
}
},
"outputs": [],
"source": [
"import ipyparallel\n",
"cluster = ipyparallel.Client(profile='pbs')"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"To check whether yt was installed correctly, simply import the framework."
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {
"collapsed": false,
"slideshow": {
"slide_type": "-"
}
},
"outputs": [
{
"data": {
"text/plain": [
"'3.4-dev'"
]
},
"execution_count": 2,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"import yt\n",
"yt.__version__"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "-"
}
},
"source": [
"For a better understanding of what happens under the hood of yt, you can change the verbosity level on the fly.\n",
"\n",
"See http://yt-project.org/doc/faq/#how-can-i-change-yt-s-log-level"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"yt.funcs.mylog.setLevel(10)"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "skip"
}
},
"source": [
"### Obtaining some example data\n",
"\n",
"```bash\n",
"#!/bin/bash\n",
"wget https://github.com/openPMD/openPMD-example-datasets/raw/draft/example-2d.tar.gz\n",
"tar xf example-2d.tar.gz\n",
"wget https://github.com/openPMD/openPMD-example-datasets/raw/draft/example-3d.tar.gz\n",
"tar xf example-3d.tar.gz\n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"### Opening files\n",
"\n",
"For single files, yt.load() does the job.\n",
"\n",
"\n"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"yt : [INFO ] 2016-09-08 23:37:45,178 open_pmd - found 21 iterations in directory\n",
"yt : [INFO ] 2016-09-08 23:37:45,207 Parameters: current_time = 1.39e-13\n",
"yt : [INFO ] 2016-09-08 23:37:45,208 Parameters: domain_dimensions = [ 80. 384. 80.]\n",
"yt : [INFO ] 2016-09-08 23:37:45,210 Parameters: domain_left_edge = [ 0. 0. 0.]\n",
"yt : [INFO ] 2016-09-08 23:37:45,212 Parameters: domain_right_edge = [ 2.08000000e-05 1.70111998e-05 2.08000000e-05]\n",
"yt : [INFO ] 2016-09-08 23:37:45,213 Parameters: cosmological_simulation = 0\n",
"yt : [DEBUG ] 2016-09-08 23:37:45,216 open_pmd - self.particle_types: ('io',)\n"
]
}
],
"source": [
"f = yt.load('/bigdata/hplsim/development/huebl/lwfa-openPMD-058-smallLWFA-h5/simOutput/h5/simData_1000.h5')"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"Generally, I'd recommend using Info (20) or Warning (30) verbosity levels. If you've made yourself familiar with yt, you may even go up to Error (40)."
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"yt.funcs.mylog.setLevel(40)"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"You can load multiple files at once in two different ways. For multiple files, both yield the same result. You may use whatever you prefer:\n",
"\n",
"yt.load() with wildcards **or**\n",
"yt.DatasetSeries() with wildcards"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {
"collapsed": false,
"scrolled": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"simData_0.h5\n",
"simData_1000.h5\n",
"simData_1250.h5\n",
"simData_1500.h5\n",
"simData_1750.h5\n",
"simData_2000.h5\n",
"simData_2250.h5\n",
"simData_250.h5\n",
"simData_2500.h5\n",
"simData_2750.h5\n",
"simData_3000.h5\n",
"simData_3250.h5\n",
"simData_3500.h5\n",
"simData_3750.h5\n",
"simData_4000.h5\n",
"simData_4250.h5\n",
"simData_4500.h5\n",
"simData_4750.h5\n",
"simData_500.h5\n",
"simData_5000.h5\n",
"simData_750.h5\n"
]
}
],
"source": [
"series = yt.load('/bigdata/hplsim/development/huebl/lwfa-openPMD-058-smallLWFA-h5/simOutput/h5/simData_*.h5')\n",
"\n",
"for handle in series:\n",
" print handle"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"### Accessing your data\n",
"\n",
"You primarily will interact with yt through fields. Both mesh and particle data are treated as fields.\n",
"`f.field_list` will give you the native on-disk fields from the loaded dataset.\n",
"\n",
"#### NOTES\n",
"1. Fields are tuples ('field_type', 'field_name')\n",
" - field type 'openPMD' represents mesh data\n",
" - all othe field types in `f.field_list` are particle data\n",
" - you will only see the on-disk particle type if there is more than one particle species\n",
" - if that is not the case, 'io' is the default particle type\n",
"2. Certain fields from your dataset will be renamed to preserve yt's intuition about the associated data\n",
" - 'position' (as in openPMD) of a particle is accessible through 'particle\\_positionCoarse\\_[x|y|z]' (in yt)\n",
" - 'particle\\_position\\_[x|y|z]' (in yt) represent a refined position calculated from the position and positionOffset from openPMD\n",
"3. Magnetic fields (independent of their name in openPMD) will get aliased to ('gas', 'magnetic\\_field\\_???') if their unitDimension matches that of T (Tesla). It is recommended to use this alias instead of the on-disk field name"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {
"collapsed": false,
"scrolled": true,
"slideshow": {
"slide_type": "slide"
}
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"/data/home/koller47/openpmd/yt/units/yt_array.py:974: RuntimeWarning: divide by zero encountered in divide\n",
" return super(YTArray, self).__div__(ro)\n"
]
},
{
"data": {
"text/plain": [
"[('all', 'particle_charge'),\n",
" ('all', 'particle_mass'),\n",
" ('all', 'particle_momentum_x'),\n",
" ('all', 'particle_momentum_y'),\n",
" ('all', 'particle_momentum_z'),\n",
" ('all', 'particle_positionCoarse_x'),\n",
" ('all', 'particle_positionCoarse_y'),\n",
" ('all', 'particle_positionCoarse_z'),\n",
" ('all', 'particle_positionOffset_x'),\n",
" ('all', 'particle_positionOffset_y'),\n",
" ('all', 'particle_positionOffset_z'),\n",
" ('all', 'particle_weighting'),\n",
" ('io', 'particle_charge'),\n",
" ('io', 'particle_mass'),\n",
" ('io', 'particle_momentum_x'),\n",
" ('io', 'particle_momentum_y'),\n",
" ('io', 'particle_momentum_z'),\n",
" ('io', 'particle_positionCoarse_x'),\n",
" ('io', 'particle_positionCoarse_y'),\n",
" ('io', 'particle_positionCoarse_z'),\n",
" ('io', 'particle_positionOffset_x'),\n",
" ('io', 'particle_positionOffset_y'),\n",
" ('io', 'particle_positionOffset_z'),\n",
" ('io', 'particle_weighting'),\n",
" ('openPMD', 'B_x'),\n",
" ('openPMD', 'B_y'),\n",
" ('openPMD', 'B_z'),\n",
" ('openPMD', 'E_x'),\n",
" ('openPMD', 'E_y'),\n",
" ('openPMD', 'E_z'),\n",
" ('openPMD', 'J_x'),\n",
" ('openPMD', 'J_y'),\n",
" ('openPMD', 'J_z'),\n",
" ('openPMD', 'e-density'),\n",
" ('openPMD', 'e-particleEnergy'),\n",
" ('openPMD', 'e-particleMomentumComponent')]"
]
},
"execution_count": 7,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"f.field_list"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {
"collapsed": false,
"slideshow": {
"slide_type": "slide"
}
},
"outputs": [
{
"data": {
"text/plain": [
"('io', 'all')"
]
},
"execution_count": 8,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"f.particle_types"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {
"collapsed": false,
"scrolled": true,
"slideshow": {
"slide_type": "slide"
}
},
"outputs": [
{
"data": {
"text/plain": [
"[('Hydrogen1+', 'particle_charge'),\n",
" ('Hydrogen1+', 'particle_mass'),\n",
" ('Hydrogen1+', 'particle_momentum_x'),\n",
" ('Hydrogen1+', 'particle_momentum_y'),\n",
" ('Hydrogen1+', 'particle_momentum_z'),\n",
" ('Hydrogen1+', 'particle_positionCoarse_x'),\n",
" ('Hydrogen1+', 'particle_positionCoarse_y'),\n",
" ('Hydrogen1+', 'particle_positionCoarse_z'),\n",
" ('Hydrogen1+', 'particle_positionOffset_x'),\n",
" ('Hydrogen1+', 'particle_positionOffset_y'),\n",
" ('Hydrogen1+', 'particle_positionOffset_z'),\n",
" ('Hydrogen1+', 'particle_weighting'),\n",
" ('all', 'particle_charge'),\n",
" ('all', 'particle_mass'),\n",
" ('all', 'particle_momentum_x'),\n",
" ('all', 'particle_momentum_y'),\n",
" ('all', 'particle_momentum_z'),\n",
" ('all', 'particle_positionCoarse_x'),\n",
" ('all', 'particle_positionCoarse_y'),\n",
" ('all', 'particle_positionCoarse_z'),\n",
" ('all', 'particle_positionOffset_x'),\n",
" ('all', 'particle_positionOffset_y'),\n",
" ('all', 'particle_positionOffset_z'),\n",
" ('all', 'particle_weighting'),\n",
" ('electrons', 'particle_charge'),\n",
" ('electrons', 'particle_mass'),\n",
" ('electrons', 'particle_momentum_x'),\n",
" ('electrons', 'particle_momentum_y'),\n",
" ('electrons', 'particle_momentum_z'),\n",
" ('electrons', 'particle_positionCoarse_x'),\n",
" ('electrons', 'particle_positionCoarse_y'),\n",
" ('electrons', 'particle_positionCoarse_z'),\n",
" ('electrons', 'particle_positionOffset_x'),\n",
" ('electrons', 'particle_positionOffset_y'),\n",
" ('electrons', 'particle_positionOffset_z'),\n",
" ('electrons', 'particle_weighting'),\n",
" ('openPMD', 'B_x'),\n",
" ('openPMD', 'B_y'),\n",
" ('openPMD', 'B_z'),\n",
" ('openPMD', 'E_x'),\n",
" ('openPMD', 'E_y'),\n",
" ('openPMD', 'E_z'),\n",
" ('openPMD', 'J_x'),\n",
" ('openPMD', 'J_y'),\n",
" ('openPMD', 'J_z'),\n",
" ('openPMD', 'rho')]"
]
},
"execution_count": 9,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"multiple_species = yt.load('~/example-2d/hdf5/data00000100.h5')\n",
"multiple_species.field_list"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {
"collapsed": false,
"slideshow": {
"slide_type": "slide"
}
},
"outputs": [
{
"data": {
"text/plain": [
"('Hydrogen1+', 'electrons', 'all')"
]
},
"execution_count": 10,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"multiple_species.particle_types"
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {
"collapsed": false,
"slideshow": {
"slide_type": "-"
}
},
"outputs": [
{
"data": {
"text/plain": [
"{'Hydrogen1+': 27400, 'electrons': 27400}"
]
},
"execution_count": 11,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"multiple_species.particle_type_counts"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"### Derived fields\n",
"\n",
"Derived fields act and look exactly like fields that exist on disk. This means that they will be generated on demand and can be used anywhere a field that exists on disk would be used. You can create them by just writing python functions.\n",
"\n",
"There _usually_ are fields automatically derived from the on-disk ones in your dataset. If your data (_read: the name and units of your fields_) fits yt's assumptions, yt does generate certain derived fields for you. You can also create these by hand from a python console.\n",
"\n",
"It is very likely you will only be concerned with field types 'all', 'openPMD' and either 'io' or your on-disk particles species.\n",
"\n",
"See http://yt-project.org/docs/dev/analyzing/fields.html for further information.\n",
"\n",
"To create a field, you define a function that takes field and data as input. You may additionally wrap this funtion inside another function if you need additional parameters (e.g. axis, threshold, ...).\n",
"\n",
"If your defined funtion does not work (i.e. it throws an exception), no derived field will be added. You should use functions from numpy instead of math wherever possible.\n",
"\n",
"See http://yt-project.org/docs/dev/developing/creating_derived_fields.html"
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {
"collapsed": false,
"scrolled": true,
"slideshow": {
"slide_type": "slide"
}
},
"outputs": [
{
"data": {
"text/plain": [
"[('all', 'mesh_id'),\n",
" ('all', 'particle_angular_momentum'),\n",
" ('all', 'particle_angular_momentum_magnitude'),\n",
" ('all', 'particle_angular_momentum_x'),\n",
" ('all', 'particle_angular_momentum_y'),\n",
" ('all', 'particle_angular_momentum_z'),\n",
" ('all', 'particle_charge'),\n",
" ('all', 'particle_cylindrical_velocity_theta'),\n",
" ('all', 'particle_cylindrical_velocity_z'),\n",
" ('all', 'particle_kinetic_energy'),\n",
" ('all', 'particle_mass'),\n",
" ('all', 'particle_momentum_x'),\n",
" ('all', 'particle_momentum_y'),\n",
" ('all', 'particle_momentum_z'),\n",
" ('all', 'particle_ones'),\n",
" ('all', 'particle_position'),\n",
" ('all', 'particle_positionCoarse_x'),\n",
" ('all', 'particle_positionCoarse_y'),\n",
" ('all', 'particle_positionCoarse_z'),\n",
" ('all', 'particle_positionOffset_x'),\n",
" ('all', 'particle_positionOffset_y'),\n",
" ('all', 'particle_positionOffset_z'),\n",
" ('all', 'particle_position_cylindrical_radius'),\n",
" ('all', 'particle_position_cylindrical_theta'),\n",
" ('all', 'particle_position_cylindrical_z'),\n",
" ('all', 'particle_position_relative'),\n",
" ('all', 'particle_position_relative_x'),\n",
" ('all', 'particle_position_relative_y'),\n",
" ('all', 'particle_position_relative_z'),\n",
" ('all', 'particle_position_spherical_phi'),\n",
" ('all', 'particle_position_spherical_theta'),\n",
" ('all', 'particle_position_x'),\n",
" ('all', 'particle_position_y'),\n",
" ('all', 'particle_position_z'),\n",
" ('all', 'particle_radial_velocity'),\n",
" ('all', 'particle_specific_angular_momentum'),\n",
" ('all', 'particle_specific_angular_momentum_x'),\n",
" ('all', 'particle_specific_angular_momentum_y'),\n",
" ('all', 'particle_specific_angular_momentum_z'),\n",
" ('all', 'particle_spherical_position_phi'),\n",
" ('all', 'particle_spherical_position_theta'),\n",
" ('all', 'particle_spherical_velocity_phi'),\n",
" ('all', 'particle_spherical_velocity_radius'),\n",
" ('all', 'particle_spherical_velocity_theta'),\n",
" ('all', 'particle_velocity'),\n",
" ('all', 'particle_velocity_cylindrical_radius'),\n",
" ('all', 'particle_velocity_cylindrical_theta'),\n",
" ('all', 'particle_velocity_cylindrical_z'),\n",
" ('all', 'particle_velocity_magnitude'),\n",
" ('all', 'particle_velocity_relative'),\n",
" ('all', 'particle_velocity_relative_x'),\n",
" ('all', 'particle_velocity_relative_y'),\n",
" ('all', 'particle_velocity_relative_z'),\n",
" ('all', 'particle_velocity_spherical_phi'),\n",
" ('all', 'particle_velocity_spherical_radius'),\n",
" ('all', 'particle_velocity_spherical_theta'),\n",
" ('all', 'particle_velocity_x'),\n",
" ('all', 'particle_velocity_y'),\n",
" ('all', 'particle_velocity_z'),\n",
" ('all', 'particle_weighting'),\n",
" ('deposit', 'all_cic'),\n",
" ('deposit', 'all_cic_velocity_x'),\n",
" ('deposit', 'all_cic_velocity_y'),\n",
" ('deposit', 'all_cic_velocity_z'),\n",
" ('deposit', 'all_count'),\n",
" ('deposit', 'all_density'),\n",
" ('deposit', 'all_mass'),\n",
" ('deposit', 'all_nn_velocity_x'),\n",
" ('deposit', 'all_nn_velocity_y'),\n",
" ('deposit', 'all_nn_velocity_z'),\n",
" ('deposit', 'io_cic'),\n",
" ('deposit', 'io_cic_velocity_x'),\n",
" ('deposit', 'io_cic_velocity_y'),\n",
" ('deposit', 'io_cic_velocity_z'),\n",
" ('deposit', 'io_count'),\n",
" ('deposit', 'io_density'),\n",
" ('deposit', 'io_mass'),\n",
" ('deposit', 'io_nn_velocity_x'),\n",
" ('deposit', 'io_nn_velocity_y'),\n",
" ('deposit', 'io_nn_velocity_z'),\n",
" ('gas', 'cell_volume'),\n",
" ('gas', 'cutting_plane_magnetic_field_x'),\n",
" ('gas', 'cutting_plane_magnetic_field_y'),\n",
" ('gas', 'cutting_plane_magnetic_field_z'),\n",
" ('gas', 'dx'),\n",
" ('gas', 'dy'),\n",
" ('gas', 'dz'),\n",
" ('gas', 'magnetic_energy'),\n",
" ('gas', 'magnetic_field_divergence'),\n",
" ('gas', 'magnetic_field_divergence_absolute'),\n",
" ('gas', 'magnetic_field_magnitude'),\n",
" ('gas', 'magnetic_field_poloidal'),\n",
" ('gas', 'magnetic_field_strength'),\n",
" ('gas', 'magnetic_field_toroidal'),\n",
" ('gas', 'magnetic_field_x'),\n",
" ('gas', 'magnetic_field_y'),\n",
" ('gas', 'magnetic_field_z'),\n",
" ('gas', 'magnetic_pressure'),\n",
" ('gas', 'path_element_x'),\n",
" ('gas', 'path_element_y'),\n",
" ('gas', 'path_element_z'),\n",
" ('gas', 'vertex_x'),\n",
" ('gas', 'vertex_y'),\n",
" ('gas', 'vertex_z'),\n",
" ('gas', 'x'),\n",
" ('gas', 'y'),\n",
" ('gas', 'z'),\n",
" ('index', 'cell_volume'),\n",
" ('index', 'cylindrical_r'),\n",
" ('index', 'cylindrical_radius'),\n",
" ('index', 'cylindrical_theta'),\n",
" ('index', 'cylindrical_z'),\n",
" ('index', 'disk_angle'),\n",
" ('index', 'dx'),\n",
" ('index', 'dy'),\n",
" ('index', 'dz'),\n",
" ('index', 'grid_indices'),\n",
" ('index', 'grid_level'),\n",
" ('index', 'height'),\n",
" ('index', 'ones'),\n",
" ('index', 'ones_over_dx'),\n",
" ('index', 'path_element_x'),\n",
" ('index', 'path_element_y'),\n",
" ('index', 'path_element_z'),\n",
" ('index', 'spherical_phi'),\n",
" ('index', 'spherical_r'),\n",
" ('index', 'spherical_radius'),\n",
" ('index', 'spherical_theta'),\n",
" ('index', 'vertex_x'),\n",
" ('index', 'vertex_y'),\n",
" ('index', 'vertex_z'),\n",
" ('index', 'x'),\n",
" ('index', 'y'),\n",
" ('index', 'z'),\n",
" ('index', 'zeros'),\n",
" ('io', 'mesh_id'),\n",
" ('io', 'particle_angular_momentum'),\n",
" ('io', 'particle_angular_momentum_magnitude'),\n",
" ('io', 'particle_angular_momentum_x'),\n",
" ('io', 'particle_angular_momentum_y'),\n",
" ('io', 'particle_angular_momentum_z'),\n",
" ('io', 'particle_charge'),\n",
" ('io', 'particle_cylindrical_velocity_theta'),\n",
" ('io', 'particle_cylindrical_velocity_z'),\n",
" ('io', 'particle_kinetic_energy'),\n",
" ('io', 'particle_mass'),\n",
" ('io', 'particle_momentum_x'),\n",
" ('io', 'particle_momentum_y'),\n",
" ('io', 'particle_momentum_z'),\n",
" ('io', 'particle_ones'),\n",
" ('io', 'particle_position'),\n",
" ('io', 'particle_positionCoarse_x'),\n",
" ('io', 'particle_positionCoarse_y'),\n",
" ('io', 'particle_positionCoarse_z'),\n",
" ('io', 'particle_positionOffset_x'),\n",
" ('io', 'particle_positionOffset_y'),\n",
" ('io', 'particle_positionOffset_z'),\n",
" ('io', 'particle_position_cylindrical_radius'),\n",
" ('io', 'particle_position_cylindrical_theta'),\n",
" ('io', 'particle_position_cylindrical_z'),\n",
" ('io', 'particle_position_relative'),\n",
" ('io', 'particle_position_relative_x'),\n",
" ('io', 'particle_position_relative_y'),\n",
" ('io', 'particle_position_relative_z'),\n",
" ('io', 'particle_position_spherical_phi'),\n",
" ('io', 'particle_position_spherical_theta'),\n",
" ('io', 'particle_position_x'),\n",
" ('io', 'particle_position_y'),\n",
" ('io', 'particle_position_z'),\n",
" ('io', 'particle_radial_velocity'),\n",
" ('io', 'particle_specific_angular_momentum'),\n",
" ('io', 'particle_specific_angular_momentum_x'),\n",
" ('io', 'particle_specific_angular_momentum_y'),\n",
" ('io', 'particle_specific_angular_momentum_z'),\n",
" ('io', 'particle_spherical_position_phi'),\n",
" ('io', 'particle_spherical_position_theta'),\n",
" ('io', 'particle_spherical_velocity_phi'),\n",
" ('io', 'particle_spherical_velocity_radius'),\n",
" ('io', 'particle_spherical_velocity_theta'),\n",
" ('io', 'particle_velocity'),\n",
" ('io', 'particle_velocity_cylindrical_radius'),\n",
" ('io', 'particle_velocity_cylindrical_theta'),\n",
" ('io', 'particle_velocity_cylindrical_z'),\n",
" ('io', 'particle_velocity_magnitude'),\n",
" ('io', 'particle_velocity_relative'),\n",
" ('io', 'particle_velocity_relative_x'),\n",
" ('io', 'particle_velocity_relative_y'),\n",
" ('io', 'particle_velocity_relative_z'),\n",
" ('io', 'particle_velocity_spherical_phi'),\n",
" ('io', 'particle_velocity_spherical_radius'),\n",
" ('io', 'particle_velocity_spherical_theta'),\n",
" ('io', 'particle_velocity_x'),\n",
" ('io', 'particle_velocity_y'),\n",
" ('io', 'particle_velocity_z'),\n",
" ('io', 'particle_weighting'),\n",
" ('openPMD', 'B_x'),\n",
" ('openPMD', 'B_y'),\n",
" ('openPMD', 'B_z'),\n",
" ('openPMD', 'E_x'),\n",
" ('openPMD', 'E_y'),\n",
" ('openPMD', 'E_z'),\n",
" ('openPMD', 'J_x'),\n",
" ('openPMD', 'J_y'),\n",
" ('openPMD', 'J_z'),\n",
" ('openPMD', 'cell_volume'),\n",
" ('openPMD', 'dx'),\n",
" ('openPMD', 'dy'),\n",
" ('openPMD', 'dz'),\n",
" ('openPMD', 'e-density'),\n",
" ('openPMD', 'e-particleEnergy'),\n",
" ('openPMD', 'e-particleMomentumComponent'),\n",
" ('openPMD', 'path_element_x'),\n",
" ('openPMD', 'path_element_y'),\n",
" ('openPMD', 'path_element_z'),\n",
" ('openPMD', 'poynting_vector_x'),\n",
" ('openPMD', 'poynting_vector_y'),\n",
" ('openPMD', 'poynting_vector_z'),\n",
" ('openPMD', 'vertex_x'),\n",
" ('openPMD', 'vertex_y'),\n",
" ('openPMD', 'vertex_z'),\n",
" ('openPMD', 'x'),\n",
" ('openPMD', 'y'),\n",
" ('openPMD', 'z')]"
]
},
"execution_count": 12,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"f.derived_field_list"
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {
"collapsed": false,
"slideshow": {
"slide_type": "slide"
}
},
"outputs": [
{
"data": {
"text/plain": [
"[('Hydrogen1+', 'mesh_id'),\n",
" ('Hydrogen1+', 'particle_charge'),\n",
" ('Hydrogen1+', 'particle_mass'),\n",
" ('Hydrogen1+', 'particle_momentum_x'),\n",
" ('Hydrogen1+', 'particle_momentum_y'),\n",
" ('Hydrogen1+', 'particle_momentum_z'),\n",
" ('Hydrogen1+', 'particle_ones'),\n",
" ('Hydrogen1+', 'particle_position'),\n",
" ('Hydrogen1+', 'particle_positionCoarse_x'),\n",
" ('Hydrogen1+', 'particle_positionCoarse_y'),\n",
" ('Hydrogen1+', 'particle_positionCoarse_z'),\n",
" ('Hydrogen1+', 'particle_positionOffset_x'),\n",
" ('Hydrogen1+', 'particle_positionOffset_y'),\n",
" ('Hydrogen1+', 'particle_positionOffset_z'),\n",
" ('Hydrogen1+', 'particle_position_cylindrical_radius'),\n",
" ('Hydrogen1+', 'particle_position_cylindrical_theta'),\n",
" ('Hydrogen1+', 'particle_position_cylindrical_z'),\n",
" ('Hydrogen1+', 'particle_position_relative'),\n",
" ('Hydrogen1+', 'particle_position_relative_x'),\n",
" ('Hydrogen1+', 'particle_position_relative_y'),\n",
" ('Hydrogen1+', 'particle_position_relative_z'),\n",
" ('Hydrogen1+', 'particle_position_spherical_phi'),\n",
" ('Hydrogen1+', 'particle_position_spherical_theta'),\n",
" ('Hydrogen1+', 'particle_position_x'),\n",
" ('Hydrogen1+', 'particle_position_y'),\n",
" ('Hydrogen1+', 'particle_position_z'),\n",
" ('Hydrogen1+', 'particle_spherical_position_phi'),\n",
" ('Hydrogen1+', 'particle_spherical_position_theta'),\n",
" ('Hydrogen1+', 'particle_weighting'),\n",
" ('all', 'mesh_id'),\n",
" ('all', 'particle_charge'),\n",
" ('all', 'particle_mass'),\n",
" ('all', 'particle_momentum_x'),\n",
" ('all', 'particle_momentum_y'),\n",
" ('all', 'particle_momentum_z'),\n",
" ('all', 'particle_ones'),\n",
" ('all', 'particle_position'),\n",
" ('all', 'particle_positionCoarse_x'),\n",
" ('all', 'particle_positionCoarse_y'),\n",
" ('all', 'particle_positionCoarse_z'),\n",
" ('all', 'particle_positionOffset_x'),\n",
" ('all', 'particle_positionOffset_y'),\n",
" ('all', 'particle_positionOffset_z'),\n",
" ('all', 'particle_position_cylindrical_radius'),\n",
" ('all', 'particle_position_cylindrical_theta'),\n",
" ('all', 'particle_position_cylindrical_z'),\n",
" ('all', 'particle_position_relative'),\n",
" ('all', 'particle_position_relative_x'),\n",
" ('all', 'particle_position_relative_y'),\n",
" ('all', 'particle_position_relative_z'),\n",
" ('all', 'particle_position_spherical_phi'),\n",
" ('all', 'particle_position_spherical_theta'),\n",
" ('all', 'particle_position_x'),\n",
" ('all', 'particle_position_y'),\n",
" ('all', 'particle_position_z'),\n",
" ('all', 'particle_spherical_position_phi'),\n",
" ('all', 'particle_spherical_position_theta'),\n",
" ('all', 'particle_weighting'),\n",
" ('deposit', 'Hydrogen1+_cic'),\n",
" ('deposit', 'Hydrogen1+_count'),\n",
" ('deposit', 'all_cic'),\n",
" ('deposit', 'all_count'),\n",
" ('deposit', 'electrons_cic'),\n",
" ('deposit', 'electrons_count'),\n",
" ('electrons', 'mesh_id'),\n",
" ('electrons', 'particle_charge'),\n",
" ('electrons', 'particle_mass'),\n",
" ('electrons', 'particle_momentum_x'),\n",
" ('electrons', 'particle_momentum_y'),\n",
" ('electrons', 'particle_momentum_z'),\n",
" ('electrons', 'particle_ones'),\n",
" ('electrons', 'particle_position'),\n",
" ('electrons', 'particle_positionCoarse_x'),\n",
" ('electrons', 'particle_positionCoarse_y'),\n",
" ('electrons', 'particle_positionCoarse_z'),\n",
" ('electrons', 'particle_positionOffset_x'),\n",
" ('electrons', 'particle_positionOffset_y'),\n",
" ('electrons', 'particle_positionOffset_z'),\n",
" ('electrons', 'particle_position_cylindrical_radius'),\n",
" ('electrons', 'particle_position_cylindrical_theta'),\n",
" ('electrons', 'particle_position_cylindrical_z'),\n",
" ('electrons', 'particle_position_relative'),\n",
" ('electrons', 'particle_position_relative_x'),\n",
" ('electrons', 'particle_position_relative_y'),\n",
" ('electrons', 'particle_position_relative_z'),\n",
" ('electrons', 'particle_position_spherical_phi'),\n",
" ('electrons', 'particle_position_spherical_theta'),\n",
" ('electrons', 'particle_position_x'),\n",
" ('electrons', 'particle_position_y'),\n",
" ('electrons', 'particle_position_z'),\n",
" ('electrons', 'particle_spherical_position_phi'),\n",
" ('electrons', 'particle_spherical_position_theta'),\n",
" ('electrons', 'particle_weighting'),\n",
" ('gas', 'cell_volume'),\n",
" ('gas', 'cutting_plane_magnetic_field_x'),\n",
" ('gas', 'cutting_plane_magnetic_field_y'),\n",
" ('gas', 'cutting_plane_magnetic_field_z'),\n",
" ('gas', 'dx'),\n",
" ('gas', 'dy'),\n",
" ('gas', 'dz'),\n",
" ('gas', 'magnetic_energy'),\n",
" ('gas', 'magnetic_field_divergence'),\n",
" ('gas', 'magnetic_field_divergence_absolute'),\n",
" ('gas', 'magnetic_field_magnitude'),\n",
" ('gas', 'magnetic_field_poloidal'),\n",
" ('gas', 'magnetic_field_strength'),\n",
" ('gas', 'magnetic_field_toroidal'),\n",
" ('gas', 'magnetic_field_x'),\n",
" ('gas', 'magnetic_field_y'),\n",
" ('gas', 'magnetic_field_z'),\n",
" ('gas', 'magnetic_pressure'),\n",
" ('gas', 'path_element_x'),\n",
" ('gas', 'path_element_y'),\n",
" ('gas', 'path_element_z'),\n",
" ('gas', 'vertex_x'),\n",
" ('gas', 'vertex_y'),\n",
" ('gas', 'vertex_z'),\n",
" ('gas', 'x'),\n",
" ('gas', 'y'),\n",
" ('gas', 'z'),\n",
" ('index', 'cell_volume'),\n",
" ('index', 'cylindrical_r'),\n",
" ('index', 'cylindrical_radius'),\n",
" ('index', 'cylindrical_theta'),\n",
" ('index', 'cylindrical_z'),\n",
" ('index', 'disk_angle'),\n",
" ('index', 'dx'),\n",
" ('index', 'dy'),\n",
" ('index', 'dz'),\n",
" ('index', 'grid_indices'),\n",
" ('index', 'grid_level'),\n",
" ('index', 'height'),\n",
" ('index', 'ones'),\n",
" ('index', 'ones_over_dx'),\n",
" ('index', 'path_element_x'),\n",
" ('index', 'path_element_y'),\n",
" ('index', 'path_element_z'),\n",
" ('index', 'spherical_phi'),\n",
" ('index', 'spherical_r'),\n",
" ('index', 'spherical_radius'),\n",
" ('index', 'spherical_theta'),\n",
" ('index', 'vertex_x'),\n",
" ('index', 'vertex_y'),\n",
" ('index', 'vertex_z'),\n",
" ('index', 'x'),\n",
" ('index', 'y'),\n",
" ('index', 'z'),\n",
" ('index', 'zeros'),\n",
" ('openPMD', 'B_x'),\n",
" ('openPMD', 'B_y'),\n",
" ('openPMD', 'B_z'),\n",
" ('openPMD', 'E_x'),\n",
" ('openPMD', 'E_y'),\n",
" ('openPMD', 'E_z'),\n",
" ('openPMD', 'J_x'),\n",
" ('openPMD', 'J_y'),\n",
" ('openPMD', 'J_z'),\n",
" ('openPMD', 'cell_volume'),\n",
" ('openPMD', 'dx'),\n",
" ('openPMD', 'dy'),\n",
" ('openPMD', 'dz'),\n",
" ('openPMD', 'path_element_x'),\n",
" ('openPMD', 'path_element_y'),\n",
" ('openPMD', 'path_element_z'),\n",
" ('openPMD', 'poynting_vector_x'),\n",
" ('openPMD', 'poynting_vector_y'),\n",
" ('openPMD', 'poynting_vector_z'),\n",
" ('openPMD', 'rho'),\n",
" ('openPMD', 'vertex_x'),\n",
" ('openPMD', 'vertex_y'),\n",
" ('openPMD', 'vertex_z'),\n",
" ('openPMD', 'x'),\n",
" ('openPMD', 'y'),\n",
" ('openPMD', 'z')]"
]
},
"execution_count": 13,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"multiple_species.derived_field_list"
]
},
{
"cell_type": "code",
"execution_count": 14,
"metadata": {
"collapsed": false,
"scrolled": true,
"slideshow": {
"slide_type": "slide"
}
},
"outputs": [
{
"data": {
"text/plain": [
"On-Disk Field (openPMD, E_x): (units: kg*m/(A*s**3))"
]
},
"execution_count": 14,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"f.field_info[('openPMD', 'E_x')]"
]
},
{
"cell_type": "code",
"execution_count": 15,
"metadata": {
"collapsed": false,
"slideshow": {
"slide_type": "-"
}
},
"outputs": [
{
"data": {
"text/plain": [
"Derived Field (io, particle_position_x): (units: m, particle field)"
]
},
"execution_count": 15,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"f.field_info[('io', 'particle_position_x')]"
]
},
{
"cell_type": "code",
"execution_count": 16,
"metadata": {
"collapsed": false,
"slideshow": {
"slide_type": "slide"
}
},
"outputs": [],
"source": [
"import numpy as np\n",
"\n",
"# These will both yield the same result (for 3D datasets)\n",
"def _magni(field, data):\n",
" return np.sqrt(data[('openPMD', 'E_x')]**2 + data[('openPMD', 'E_y')]**2 + data[('openPMD', 'E_z')]**2)\n",
"\n",
"def _limited_magni(num_axis):\n",
" def _lm(field, data):\n",
" if num_axis is 1:\n",
" return np.sqrt(data[('openPMD', 'E_x')]**2)\n",
" elif num_axis is 2:\n",
" return np.sqrt(data[('openPMD', 'E_x')]**2 + data[('openPMD', 'E_y')]**2)\n",
" elif num_axis is 3:\n",
" return np.sqrt(data[('openPMD', 'E_x')]**2 + data[('openPMD', 'E_y')]**2 + data[('openPMD', 'E_z')]**2)\n",
"\n",
" return _lm"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"You can now either add the function to a specific dataset `f.add_field()` or apply it to every dataset in this session `yt.add_field()`"
]
},
{
"cell_type": "code",
"execution_count": 17,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"# These will both yield the same result (for 3D datasets)\n",
"\n",
"f.add_field(name=('openPMD', 'E_magnitude'), \n",
" function=_limited_magni(f.dimensionality), \n",
" units=\"kg*m/(A*s**3)\", \n",
" force_override=True)\n",
"\n",
"f.add_field(name=('openPMD', 'E_magnitude'), \n",
" function=_magni, \n",
" units=\"kg*m/(A*s**3)\", \n",
" force_override=True)"
]
},
{
"cell_type": "code",
"execution_count": 18,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/plain": [
"Derived Field (openPMD, E_magnitude): (units: kg*m/(A*s**3))"
]
},
"execution_count": 18,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"f.field_info[('openPMD', 'E_magnitude')]"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"Should you encounter unexpected results, you may be interested in how a derived field is calculated:"
]
},
{
"cell_type": "code",
"execution_count": 19,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"def _magni(field, data):\n",
" return np.sqrt(data[('openPMD', 'E_x')]**2 + data[('openPMD', 'E_y')]**2 + data[('openPMD', 'E_z')]**2)\n",
"\n"
]
}
],
"source": [
"print f.field_info[('openPMD', 'E_magnitude')].get_source()"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"Depending on the size of your data, it might be sliced into chunks for efficient processing. The example data provided above consists of a single particle species with 85625 unique particles and 2 meshes of size [26, 26, 201] (i.e. 135876 gridpoints each). This only results in a single chunk/grid.\n",
"\n",
"The lwfa files used below consist of 2.5M unique particles in 4 particle patches. For every such patch, a grid is generated. If the number of particles per patch exceeds a limit (`open_pmd_virtual_gridsize`), they will chunked even more.\n",
"\n",
"You will see grids up to a few million particles and gridpoints.\n",
"\n",
"Grids are what allows yt to process data from a single timestep/file in parallel."
]
},
{
"cell_type": "code",
"execution_count": 20,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/plain": [
"array([OpenPMDGrid_0000 ([ 80 384 80]), OpenPMDGrid_0001 ([0 0 0]),\n",
" OpenPMDGrid_0002 ([0 0 0]), OpenPMDGrid_0003 ([0 0 0]),\n",
" OpenPMDGrid_0004 ([0 0 0])], dtype=object)"
]
},
"execution_count": 20,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"f.index.grids"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"### Analyzing mesh data\n",
"\n",
"For plotting (binning/analyzing) mesh data, you have a range of different options:\n",
"- Slice Plots along a given axis (x, y, z or any arbitrary axis) through a center\n",
"- Projection Plots along a given axis (x, y, z or any arbitrary axis) with multiple projection methods\n",
"- Profile and Phase Plots between multiple quantities in 1D and 2D\n",
"- Volume rendering / raytracing that evaluates the radiative transfer equations through a volume with user-defined transfer functions for each ray"
]
},
{
"cell_type": "code",
"execution_count": 21,
"metadata": {
"collapsed": false,
"scrolled": false,
"slideshow": {
"slide_type": "slide"
}
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"/data/home/koller47/openpmd/yt/units/yt_array.py:1162: VisibleDeprecationWarning: using a non-integer number instead of an integer will result in an error in the future\n",
" ret = super(YTArray, self).__getitem__(item)\n"
]
},
{
"data": {
"text/html": [
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment