import sys
sys.path.append('../../../tile-benchmarking/helpers/')
import eodc_hub_role
credentials = eodc_hub_role.fetch_and_set_credentials()Tile Server End to End Benchmarks
In addition to application code benchmarks, it is important have e2e benchmarks which demonstrate the performance of the tile server. Running end-to-end benchmarks is documented in https://github.com/developmentseed/tile-benchmarking/tree/main/03-e2e/README.md.
Tested tiles ranged from zoom 0 to zoom 5. From testing we noticed that the most important factors in determining response time were the number of coordinate chunks and the chunk size. The time reported is the median response time and represents the median response time for multiple requests (usually 10) for the same tile (i.e. x, y, and z parameters).
Based on this, the recommendation would be to create chunks as small as possible (e.g. 1 timestep per chunk for the full spatial extent if visualization is the main goal) and not to chunk coordinate data.
Below, we include an example of how to plot results from one execution of these benchmarks.
First we download the results:
%%capture
# download results from s3 or copy them from the tile-benchmarking repo
!aws s3 cp --recursive s3://nasa-eodc-data-store/tile-benchmarking-results/2023-10-08_00-15-09/ downloaded_results/# Import libraries
import os
import pandas as pd
import hvplot.pandas
import holoviews as hv
pd.options.plotting.backend = 'holoviews'
import warnings
warnings.filterwarnings('ignore')Parse and merge results into a single dataframe.
# Specify the directory path and the suffix
directory_path = "downloaded_results/"
suffix = "_urls_stats.csv" # For example, if you're interested in text files
# List all files in the directory
all_files = os.listdir(directory_path)
# Filter the files to only include those that end with the specified suffix
files_with_suffix = [f"{directory_path}{f}" for f in all_files if f.endswith(suffix)]
dfs = []
for file in files_with_suffix:
df = pd.read_csv(file)
df['file'] = file
dfs.append(df)
merged_df = pd.concat(dfs)The “Name” column is either the URL path (i.e. /0/0/0) OR “Aggregated”. The “Aggregated” results represent aggregations across tile endpoints. We want to understand results by zoom level, so the “Aggregated” rows are removed and we parse the zoom from the tile endpoint. We also parse the dataset name from the file name.
df_filtered = merged_df[merged_df['Name'] != 'Aggregated']
df_filtered['zoom'] = [int(path.split('/')[2]) for path in df_filtered['Name']]
df_filtered['dataset'] = [file.split('/')[1].replace('_urls_stats.csv', '') for file in df_filtered['file']]Next, we load and merge the zoom_info.csv file that was generated when generating the URLs to test (see gen_test_urls.py, so that we can view performance alongside characterstics of the datset, like chunk size.
df_filtered_min = df_filtered[['dataset', 'zoom', 'Median Response Time']]
zarr_info = pd.read_csv('../../../tile-benchmarking/03-e2e/zarr_info.csv')
zarr_info_min = zarr_info[['collection_name', 'shape_dict', 'chunks', 'chunk_size_mb', 'number_of_spatial_chunks', 'dtype']]
# power_901_monthly_meteorology_utc is a bit of an outlier
zarr_info_min = zarr_info_min[zarr_info_min['collection_name'] != 'power_901_monthly_meteorology_utc.zarr']
results_info_merged = zarr_info_min.merge(df_filtered_min, left_on='collection_name', right_on='dataset', how='outer')Plotting results
We can plot results to see how, at low zoom levels, time is a function of both number of spatial chunks AND chunk size. As zoom levels increase, the number of spatial chunks matters less, where as the chunk size continues to impact peformance.
zoom_levels=6
plots_dict = {}
for zoom in range(zoom_levels):
subset = results_info_merged[results_info_merged['zoom'] == zoom].drop(columns=['dataset'])
chunk_plot = subset.hvplot.scatter(x='chunk_size_mb', y='Median Response Time', title=f'Zoom Level {zoom}: Time vs. Chunk Size', by='collection_name', legend=False)
# Time vs. Number of Spatial Chunks
spatial_plot = subset.hvplot.scatter(x='number_of_spatial_chunks', y='Median Response Time', title=f'Zoom Level {zoom}: Time vs. Number of Spatial Chunks', by='collection_name', legend=False)
plots_dict[zoom] = (chunk_plot + spatial_plot)
grid_plot = hv.Layout(plots_dict.values()).cols(2).opts(shared_axes=False)
grid_plotTable of responses
It may be easier to understand these results in a table. Because multiple tiles were tested above zoom 0 (which only has one tile) we can take the average response across tiles at a given zoom level and put those results in a talbe alongside the dependent variables of chunk size and number of spatial chunks for easier interpretation.
for zoom in range(6):
display(f"Results for Zoom {zoom}")
this_zoom_results = results_info_merged[results_info_merged['zoom'] == zoom].drop(columns=['dataset'])
average_response = this_zoom_results.groupby('collection_name')['Median Response Time'].mean().reset_index()
average_response.rename(columns={'Median Response Time': 'Average Median Response'}, inplace=True)
# Merge this with the original dataframe
result = pd.merge(average_response, zarr_info_min, on='collection_name', how='left')
display(result.sort_values(['Average Median Response'])) 'Results for Zoom 0'| collection_name | Average Median Response | shape_dict | chunks | chunk_size_mb | number_of_spatial_chunks | dtype | |
|---|---|---|---|---|---|---|---|
| 10 | single_chunk_store_lat512_lon1024.zarr | 220.000000 | {'time': 1, 'lat': 512, 'lon': 1024} | {'time': 1, 'lat': 512, 'lon': 1024} | 4.000000 | 1.000000 | float64 |
| 1 | 600_1440_1_CMIP6_daily_GISS-E2-1-G_tas.zarr | 270.000000 | {'time': 730, 'lat': 600, 'lon': 1440} | {'time': 1, 'lat': 600, 'lon': 1440} | 3.295898 | 1.000000 | float32 |
| 11 | single_chunk_store_lat724_lon1448.zarr | 280.000000 | {'time': 1, 'lat': 724, 'lon': 1448} | {'time': 1, 'lat': 724, 'lon': 1448} | 7.998291 | 1.000000 | float64 |
| 3 | cmip6-kerchunk | 330.000000 | {'time': 730, 'lat': 600, 'lon': 1440} | {'time': 1, 'lat': 600, 'lon': 1440} | 3.295898 | 1.000000 | float32 |
| 5 | single_chunk_store_lat1024_lon2048.zarr | 350.000000 | {'time': 1, 'lat': 1024, 'lon': 2048} | {'time': 1, 'lat': 1024, 'lon': 2048} | 16.000000 | 1.000000 | float64 |
| 4 | cmip6-pds_GISS-E2-1-G_historical_tas | 510.000000 | {'time': 1980, 'lat': 90, 'lon': 144} | {'time': 600, 'lat': 90, 'lon': 144} | 29.663086 | 1.000000 | float32 |
| 6 | single_chunk_store_lat1448_lon2896.zarr | 510.000000 | {'time': 1, 'lat': 1448, 'lon': 2896} | {'time': 1, 'lat': 1448, 'lon': 2896} | 31.993164 | 1.000000 | float64 |
| 2 | 600_1440_29_CMIP6_daily_GISS-E2-1-G_tas.zarr | 540.000000 | {'time': 730, 'lat': 600, 'lon': 1440} | {'time': 29, 'lat': 600, 'lon': 1440} | 95.581055 | 1.000000 | float32 |
| 12 | with_chunks_store_lat1448_lon2896.zarr | 550.000000 | {'time': 1, 'lat': 1448, 'lon': 2896} | {'time': 1, 'lat': 1448, 'lon': 2896} | 31.993164 | 1.000000 | float64 |
| 13 | with_chunks_store_lat2048_lon4096.zarr | 630.000000 | {'time': 1, 'lat': 2048, 'lon': 4096} | {'time': 1, 'lat': 1448, 'lon': 2896} | 31.993164 | 2.000427 | float64 |
| 14 | with_chunks_store_lat2896_lon5792.zarr | 840.000000 | {'time': 1, 'lat': 2896, 'lon': 5792} | {'time': 1, 'lat': 1448, 'lon': 2896} | 31.993164 | 4.000000 | float64 |
| 7 | single_chunk_store_lat2048_lon4096.zarr | 980.000000 | {'time': 1, 'lat': 2048, 'lon': 4096} | {'time': 1, 'lat': 2048, 'lon': 4096} | 64.000000 | 1.000000 | float64 |
| 8 | single_chunk_store_lat2896_lon5792.zarr | 1704.102722 | {'time': 1, 'lat': 2896, 'lon': 5792} | {'time': 1, 'lat': 2896, 'lon': 5792} | 127.972656 | 1.000000 | float64 |
| 15 | with_chunks_store_lat4096_lon8192.zarr | 1900.000000 | {'time': 1, 'lat': 4096, 'lon': 8192} | {'time': 1, 'lat': 1448, 'lon': 2896} | 31.993164 | 8.001709 | float64 |
| 0 | 365_262_262_CMIP6_daily_GISS-E2-1-G_tas.zarr | 3200.000000 | {'time': 730, 'lat': 600, 'lon': 1440} | {'time': 365, 'lat': 262, 'lon': 262} | 95.577469 | 12.586679 | float32 |
| 9 | single_chunk_store_lat4096_lon8192.zarr | 3700.000000 | {'time': 1, 'lat': 4096, 'lon': 8192} | {'time': 1, 'lat': 4096, 'lon': 8192} | 256.000000 | 1.000000 | float64 |
| 16 | with_chunks_store_lat5793_lon11586.zarr | 5700.000000 | {'time': 1, 'lat': 5793, 'lon': 11586} | {'time': 1, 'lat': 1448, 'lon': 2896} | 31.993164 | 16.005525 | float64 |
'Results for Zoom 1'| collection_name | Average Median Response | shape_dict | chunks | chunk_size_mb | number_of_spatial_chunks | dtype | |
|---|---|---|---|---|---|---|---|
| 10 | single_chunk_store_lat512_lon1024.zarr | 220.0 | {'time': 1, 'lat': 512, 'lon': 1024} | {'time': 1, 'lat': 512, 'lon': 1024} | 4.000000 | 1.000000 | float64 |
| 3 | cmip6-kerchunk | 225.0 | {'time': 730, 'lat': 600, 'lon': 1440} | {'time': 1, 'lat': 600, 'lon': 1440} | 3.295898 | 1.000000 | float32 |
| 11 | single_chunk_store_lat724_lon1448.zarr | 250.0 | {'time': 1, 'lat': 724, 'lon': 1448} | {'time': 1, 'lat': 724, 'lon': 1448} | 7.998291 | 1.000000 | float64 |
| 1 | 600_1440_1_CMIP6_daily_GISS-E2-1-G_tas.zarr | 260.0 | {'time': 730, 'lat': 600, 'lon': 1440} | {'time': 1, 'lat': 600, 'lon': 1440} | 3.295898 | 1.000000 | float32 |
| 5 | single_chunk_store_lat1024_lon2048.zarr | 330.0 | {'time': 1, 'lat': 1024, 'lon': 2048} | {'time': 1, 'lat': 1024, 'lon': 2048} | 16.000000 | 1.000000 | float64 |
| 6 | single_chunk_store_lat1448_lon2896.zarr | 485.0 | {'time': 1, 'lat': 1448, 'lon': 2896} | {'time': 1, 'lat': 1448, 'lon': 2896} | 31.993164 | 1.000000 | float64 |
| 12 | with_chunks_store_lat1448_lon2896.zarr | 490.0 | {'time': 1, 'lat': 1448, 'lon': 2896} | {'time': 1, 'lat': 1448, 'lon': 2896} | 31.993164 | 1.000000 | float64 |
| 4 | cmip6-pds_GISS-E2-1-G_historical_tas | 500.0 | {'time': 1980, 'lat': 90, 'lon': 144} | {'time': 600, 'lat': 90, 'lon': 144} | 29.663086 | 1.000000 | float32 |
| 13 | with_chunks_store_lat2048_lon4096.zarr | 520.0 | {'time': 1, 'lat': 2048, 'lon': 4096} | {'time': 1, 'lat': 1448, 'lon': 2896} | 31.993164 | 2.000427 | float64 |
| 2 | 600_1440_29_CMIP6_daily_GISS-E2-1-G_tas.zarr | 540.0 | {'time': 730, 'lat': 600, 'lon': 1440} | {'time': 29, 'lat': 600, 'lon': 1440} | 95.581055 | 1.000000 | float32 |
| 14 | with_chunks_store_lat2896_lon5792.zarr | 610.0 | {'time': 1, 'lat': 2896, 'lon': 5792} | {'time': 1, 'lat': 1448, 'lon': 2896} | 31.993164 | 4.000000 | float64 |
| 7 | single_chunk_store_lat2048_lon4096.zarr | 845.0 | {'time': 1, 'lat': 2048, 'lon': 4096} | {'time': 1, 'lat': 2048, 'lon': 4096} | 64.000000 | 1.000000 | float64 |
| 0 | 365_262_262_CMIP6_daily_GISS-E2-1-G_tas.zarr | 1030.0 | {'time': 730, 'lat': 600, 'lon': 1440} | {'time': 365, 'lat': 262, 'lon': 262} | 95.577469 | 12.586679 | float32 |
| 15 | with_chunks_store_lat4096_lon8192.zarr | 1115.0 | {'time': 1, 'lat': 4096, 'lon': 8192} | {'time': 1, 'lat': 1448, 'lon': 2896} | 31.993164 | 8.001709 | float64 |
| 16 | with_chunks_store_lat5793_lon11586.zarr | 1300.0 | {'time': 1, 'lat': 5793, 'lon': 11586} | {'time': 1, 'lat': 1448, 'lon': 2896} | 31.993164 | 16.005525 | float64 |
| 8 | single_chunk_store_lat2896_lon5792.zarr | 1500.0 | {'time': 1, 'lat': 2896, 'lon': 5792} | {'time': 1, 'lat': 2896, 'lon': 5792} | 127.972656 | 1.000000 | float64 |
| 9 | single_chunk_store_lat4096_lon8192.zarr | 2900.0 | {'time': 1, 'lat': 4096, 'lon': 8192} | {'time': 1, 'lat': 4096, 'lon': 8192} | 256.000000 | 1.000000 | float64 |
'Results for Zoom 2'| collection_name | Average Median Response | shape_dict | chunks | chunk_size_mb | number_of_spatial_chunks | dtype | |
|---|---|---|---|---|---|---|---|
| 3 | cmip6-kerchunk | 218.000000 | {'time': 730, 'lat': 600, 'lon': 1440} | {'time': 1, 'lat': 600, 'lon': 1440} | 3.295898 | 1.000000 | float32 |
| 10 | single_chunk_store_lat512_lon1024.zarr | 220.000000 | {'time': 1, 'lat': 512, 'lon': 1024} | {'time': 1, 'lat': 512, 'lon': 1024} | 4.000000 | 1.000000 | float64 |
| 1 | 600_1440_1_CMIP6_daily_GISS-E2-1-G_tas.zarr | 254.000000 | {'time': 730, 'lat': 600, 'lon': 1440} | {'time': 1, 'lat': 600, 'lon': 1440} | 3.295898 | 1.000000 | float32 |
| 11 | single_chunk_store_lat724_lon1448.zarr | 258.333333 | {'time': 1, 'lat': 724, 'lon': 1448} | {'time': 1, 'lat': 724, 'lon': 1448} | 7.998291 | 1.000000 | float64 |
| 5 | single_chunk_store_lat1024_lon2048.zarr | 325.000000 | {'time': 1, 'lat': 1024, 'lon': 2048} | {'time': 1, 'lat': 1024, 'lon': 2048} | 16.000000 | 1.000000 | float64 |
| 13 | with_chunks_store_lat2048_lon4096.zarr | 408.333333 | {'time': 1, 'lat': 2048, 'lon': 4096} | {'time': 1, 'lat': 1448, 'lon': 2896} | 31.993164 | 2.000427 | float64 |
| 6 | single_chunk_store_lat1448_lon2896.zarr | 482.000000 | {'time': 1, 'lat': 1448, 'lon': 2896} | {'time': 1, 'lat': 1448, 'lon': 2896} | 31.993164 | 1.000000 | float64 |
| 4 | cmip6-pds_GISS-E2-1-G_historical_tas | 486.666667 | {'time': 1980, 'lat': 90, 'lon': 144} | {'time': 600, 'lat': 90, 'lon': 144} | 29.663086 | 1.000000 | float32 |
| 12 | with_chunks_store_lat1448_lon2896.zarr | 486.666667 | {'time': 1, 'lat': 1448, 'lon': 2896} | {'time': 1, 'lat': 1448, 'lon': 2896} | 31.993164 | 1.000000 | float64 |
| 2 | 600_1440_29_CMIP6_daily_GISS-E2-1-G_tas.zarr | 510.000000 | {'time': 730, 'lat': 600, 'lon': 1440} | {'time': 29, 'lat': 600, 'lon': 1440} | 95.581055 | 1.000000 | float32 |
| 14 | with_chunks_store_lat2896_lon5792.zarr | 610.000000 | {'time': 1, 'lat': 2896, 'lon': 5792} | {'time': 1, 'lat': 1448, 'lon': 2896} | 31.993164 | 4.000000 | float64 |
| 15 | with_chunks_store_lat4096_lon8192.zarr | 817.500000 | {'time': 1, 'lat': 4096, 'lon': 8192} | {'time': 1, 'lat': 1448, 'lon': 2896} | 31.993164 | 8.001709 | float64 |
| 7 | single_chunk_store_lat2048_lon4096.zarr | 842.500000 | {'time': 1, 'lat': 2048, 'lon': 4096} | {'time': 1, 'lat': 2048, 'lon': 4096} | 64.000000 | 1.000000 | float64 |
| 0 | 365_262_262_CMIP6_daily_GISS-E2-1-G_tas.zarr | 852.500000 | {'time': 730, 'lat': 600, 'lon': 1440} | {'time': 365, 'lat': 262, 'lon': 262} | 95.577469 | 12.586679 | float32 |
| 16 | with_chunks_store_lat5793_lon11586.zarr | 1118.333333 | {'time': 1, 'lat': 5793, 'lon': 11586} | {'time': 1, 'lat': 1448, 'lon': 2896} | 31.993164 | 16.005525 | float64 |
| 8 | single_chunk_store_lat2896_lon5792.zarr | 1500.000000 | {'time': 1, 'lat': 2896, 'lon': 5792} | {'time': 1, 'lat': 2896, 'lon': 5792} | 127.972656 | 1.000000 | float64 |
| 9 | single_chunk_store_lat4096_lon8192.zarr | 2975.000000 | {'time': 1, 'lat': 4096, 'lon': 8192} | {'time': 1, 'lat': 4096, 'lon': 8192} | 256.000000 | 1.000000 | float64 |
'Results for Zoom 3'| collection_name | Average Median Response | shape_dict | chunks | chunk_size_mb | number_of_spatial_chunks | dtype | |
|---|---|---|---|---|---|---|---|
| 3 | cmip6-kerchunk | 208.181818 | {'time': 730, 'lat': 600, 'lon': 1440} | {'time': 1, 'lat': 600, 'lon': 1440} | 3.295898 | 1.000000 | float32 |
| 10 | single_chunk_store_lat512_lon1024.zarr | 214.545455 | {'time': 1, 'lat': 512, 'lon': 1024} | {'time': 1, 'lat': 512, 'lon': 1024} | 4.000000 | 1.000000 | float64 |
| 1 | 600_1440_1_CMIP6_daily_GISS-E2-1-G_tas.zarr | 245.454545 | {'time': 730, 'lat': 600, 'lon': 1440} | {'time': 1, 'lat': 600, 'lon': 1440} | 3.295898 | 1.000000 | float32 |
| 11 | single_chunk_store_lat724_lon1448.zarr | 255.833333 | {'time': 1, 'lat': 724, 'lon': 1448} | {'time': 1, 'lat': 724, 'lon': 1448} | 7.998291 | 1.000000 | float64 |
| 5 | single_chunk_store_lat1024_lon2048.zarr | 326.666667 | {'time': 1, 'lat': 1024, 'lon': 2048} | {'time': 1, 'lat': 1024, 'lon': 2048} | 16.000000 | 1.000000 | float64 |
| 13 | with_chunks_store_lat2048_lon4096.zarr | 406.363636 | {'time': 1, 'lat': 2048, 'lon': 4096} | {'time': 1, 'lat': 1448, 'lon': 2896} | 31.993164 | 2.000427 | float64 |
| 6 | single_chunk_store_lat1448_lon2896.zarr | 484.545455 | {'time': 1, 'lat': 1448, 'lon': 2896} | {'time': 1, 'lat': 1448, 'lon': 2896} | 31.993164 | 1.000000 | float64 |
| 12 | with_chunks_store_lat1448_lon2896.zarr | 484.545455 | {'time': 1, 'lat': 1448, 'lon': 2896} | {'time': 1, 'lat': 1448, 'lon': 2896} | 31.993164 | 1.000000 | float64 |
| 14 | with_chunks_store_lat2896_lon5792.zarr | 485.555556 | {'time': 1, 'lat': 2896, 'lon': 5792} | {'time': 1, 'lat': 1448, 'lon': 2896} | 31.993164 | 4.000000 | float64 |
| 4 | cmip6-pds_GISS-E2-1-G_historical_tas | 497.272727 | {'time': 1980, 'lat': 90, 'lon': 144} | {'time': 600, 'lat': 90, 'lon': 144} | 29.663086 | 1.000000 | float32 |
| 2 | 600_1440_29_CMIP6_daily_GISS-E2-1-G_tas.zarr | 512.000000 | {'time': 730, 'lat': 600, 'lon': 1440} | {'time': 29, 'lat': 600, 'lon': 1440} | 95.581055 | 1.000000 | float32 |
| 15 | with_chunks_store_lat4096_lon8192.zarr | 554.166667 | {'time': 1, 'lat': 4096, 'lon': 8192} | {'time': 1, 'lat': 1448, 'lon': 2896} | 31.993164 | 8.001709 | float64 |
| 0 | 365_262_262_CMIP6_daily_GISS-E2-1-G_tas.zarr | 647.272727 | {'time': 730, 'lat': 600, 'lon': 1440} | {'time': 365, 'lat': 262, 'lon': 262} | 95.577469 | 12.586679 | float32 |
| 16 | with_chunks_store_lat5793_lon11586.zarr | 655.000000 | {'time': 1, 'lat': 5793, 'lon': 11586} | {'time': 1, 'lat': 1448, 'lon': 2896} | 31.993164 | 16.005525 | float64 |
| 7 | single_chunk_store_lat2048_lon4096.zarr | 838.333333 | {'time': 1, 'lat': 2048, 'lon': 4096} | {'time': 1, 'lat': 2048, 'lon': 4096} | 64.000000 | 1.000000 | float64 |
| 8 | single_chunk_store_lat2896_lon5792.zarr | 1500.000000 | {'time': 1, 'lat': 2896, 'lon': 5792} | {'time': 1, 'lat': 2896, 'lon': 5792} | 127.972656 | 1.000000 | float64 |
| 9 | single_chunk_store_lat4096_lon8192.zarr | 3037.671648 | {'time': 1, 'lat': 4096, 'lon': 8192} | {'time': 1, 'lat': 4096, 'lon': 8192} | 256.000000 | 1.000000 | float64 |
'Results for Zoom 4'| collection_name | Average Median Response | shape_dict | chunks | chunk_size_mb | number_of_spatial_chunks | dtype | |
|---|---|---|---|---|---|---|---|
| 3 | cmip6-kerchunk | 200.666667 | {'time': 730, 'lat': 600, 'lon': 1440} | {'time': 1, 'lat': 600, 'lon': 1440} | 3.295898 | 1.000000 | float32 |
| 10 | single_chunk_store_lat512_lon1024.zarr | 214.666667 | {'time': 1, 'lat': 512, 'lon': 1024} | {'time': 1, 'lat': 512, 'lon': 1024} | 4.000000 | 1.000000 | float64 |
| 1 | 600_1440_1_CMIP6_daily_GISS-E2-1-G_tas.zarr | 240.000000 | {'time': 730, 'lat': 600, 'lon': 1440} | {'time': 1, 'lat': 600, 'lon': 1440} | 3.295898 | 1.000000 | float32 |
| 11 | single_chunk_store_lat724_lon1448.zarr | 253.571429 | {'time': 1, 'lat': 724, 'lon': 1448} | {'time': 1, 'lat': 724, 'lon': 1448} | 7.998291 | 1.000000 | float64 |
| 5 | single_chunk_store_lat1024_lon2048.zarr | 328.461538 | {'time': 1, 'lat': 1024, 'lon': 2048} | {'time': 1, 'lat': 1024, 'lon': 2048} | 16.000000 | 1.000000 | float64 |
| 13 | with_chunks_store_lat2048_lon4096.zarr | 410.000000 | {'time': 1, 'lat': 2048, 'lon': 4096} | {'time': 1, 'lat': 1448, 'lon': 2896} | 31.993164 | 2.000427 | float64 |
| 15 | with_chunks_store_lat4096_lon8192.zarr | 453.571429 | {'time': 1, 'lat': 4096, 'lon': 8192} | {'time': 1, 'lat': 1448, 'lon': 2896} | 31.993164 | 8.001709 | float64 |
| 6 | single_chunk_store_lat1448_lon2896.zarr | 479.333333 | {'time': 1, 'lat': 1448, 'lon': 2896} | {'time': 1, 'lat': 1448, 'lon': 2896} | 31.993164 | 1.000000 | float64 |
| 14 | with_chunks_store_lat2896_lon5792.zarr | 480.666667 | {'time': 1, 'lat': 2896, 'lon': 5792} | {'time': 1, 'lat': 1448, 'lon': 2896} | 31.993164 | 4.000000 | float64 |
| 12 | with_chunks_store_lat1448_lon2896.zarr | 486.428571 | {'time': 1, 'lat': 1448, 'lon': 2896} | {'time': 1, 'lat': 1448, 'lon': 2896} | 31.993164 | 1.000000 | float64 |
| 4 | cmip6-pds_GISS-E2-1-G_historical_tas | 504.285714 | {'time': 1980, 'lat': 90, 'lon': 144} | {'time': 600, 'lat': 90, 'lon': 144} | 29.663086 | 1.000000 | float32 |
| 2 | 600_1440_29_CMIP6_daily_GISS-E2-1-G_tas.zarr | 509.333333 | {'time': 730, 'lat': 600, 'lon': 1440} | {'time': 29, 'lat': 600, 'lon': 1440} | 95.581055 | 1.000000 | float32 |
| 16 | with_chunks_store_lat5793_lon11586.zarr | 532.666667 | {'time': 1, 'lat': 5793, 'lon': 11586} | {'time': 1, 'lat': 1448, 'lon': 2896} | 31.993164 | 16.005525 | float64 |
| 0 | 365_262_262_CMIP6_daily_GISS-E2-1-G_tas.zarr | 577.500000 | {'time': 730, 'lat': 600, 'lon': 1440} | {'time': 365, 'lat': 262, 'lon': 262} | 95.577469 | 12.586679 | float32 |
| 7 | single_chunk_store_lat2048_lon4096.zarr | 838.666667 | {'time': 1, 'lat': 2048, 'lon': 4096} | {'time': 1, 'lat': 2048, 'lon': 4096} | 64.000000 | 1.000000 | float64 |
| 8 | single_chunk_store_lat2896_lon5792.zarr | 1500.000000 | {'time': 1, 'lat': 2896, 'lon': 5792} | {'time': 1, 'lat': 2896, 'lon': 5792} | 127.972656 | 1.000000 | float64 |
| 9 | single_chunk_store_lat4096_lon8192.zarr | 3008.821526 | {'time': 1, 'lat': 4096, 'lon': 8192} | {'time': 1, 'lat': 4096, 'lon': 8192} | 256.000000 | 1.000000 | float64 |
'Results for Zoom 5'| collection_name | Average Median Response | shape_dict | chunks | chunk_size_mb | number_of_spatial_chunks | dtype | |
|---|---|---|---|---|---|---|---|
| 3 | cmip6-kerchunk | 208.148148 | {'time': 730, 'lat': 600, 'lon': 1440} | {'time': 1, 'lat': 600, 'lon': 1440} | 3.295898 | 1.000000 | float32 |
| 10 | single_chunk_store_lat512_lon1024.zarr | 214.444444 | {'time': 1, 'lat': 512, 'lon': 1024} | {'time': 1, 'lat': 512, 'lon': 1024} | 4.000000 | 1.000000 | float64 |
| 1 | 600_1440_1_CMIP6_daily_GISS-E2-1-G_tas.zarr | 240.740741 | {'time': 730, 'lat': 600, 'lon': 1440} | {'time': 1, 'lat': 600, 'lon': 1440} | 3.295898 | 1.000000 | float32 |
| 11 | single_chunk_store_lat724_lon1448.zarr | 251.851852 | {'time': 1, 'lat': 724, 'lon': 1448} | {'time': 1, 'lat': 724, 'lon': 1448} | 7.998291 | 1.000000 | float64 |
| 5 | single_chunk_store_lat1024_lon2048.zarr | 330.740741 | {'time': 1, 'lat': 1024, 'lon': 2048} | {'time': 1, 'lat': 1024, 'lon': 2048} | 16.000000 | 1.000000 | float64 |
| 13 | with_chunks_store_lat2048_lon4096.zarr | 393.703704 | {'time': 1, 'lat': 2048, 'lon': 4096} | {'time': 1, 'lat': 1448, 'lon': 2896} | 31.993164 | 2.000427 | float64 |
| 15 | with_chunks_store_lat4096_lon8192.zarr | 461.111111 | {'time': 1, 'lat': 4096, 'lon': 8192} | {'time': 1, 'lat': 1448, 'lon': 2896} | 31.993164 | 8.001709 | float64 |
| 6 | single_chunk_store_lat1448_lon2896.zarr | 480.370370 | {'time': 1, 'lat': 1448, 'lon': 2896} | {'time': 1, 'lat': 1448, 'lon': 2896} | 31.993164 | 1.000000 | float64 |
| 14 | with_chunks_store_lat2896_lon5792.zarr | 482.800000 | {'time': 1, 'lat': 2896, 'lon': 5792} | {'time': 1, 'lat': 1448, 'lon': 2896} | 31.993164 | 4.000000 | float64 |
| 12 | with_chunks_store_lat1448_lon2896.zarr | 483.461538 | {'time': 1, 'lat': 1448, 'lon': 2896} | {'time': 1, 'lat': 1448, 'lon': 2896} | 31.993164 | 1.000000 | float64 |
| 16 | with_chunks_store_lat5793_lon11586.zarr | 493.461538 | {'time': 1, 'lat': 5793, 'lon': 11586} | {'time': 1, 'lat': 1448, 'lon': 2896} | 31.993164 | 16.005525 | float64 |
| 4 | cmip6-pds_GISS-E2-1-G_historical_tas | 497.777778 | {'time': 1980, 'lat': 90, 'lon': 144} | {'time': 600, 'lat': 90, 'lon': 144} | 29.663086 | 1.000000 | float32 |
| 2 | 600_1440_29_CMIP6_daily_GISS-E2-1-G_tas.zarr | 511.851852 | {'time': 730, 'lat': 600, 'lon': 1440} | {'time': 29, 'lat': 600, 'lon': 1440} | 95.581055 | 1.000000 | float32 |
| 0 | 365_262_262_CMIP6_daily_GISS-E2-1-G_tas.zarr | 614.814815 | {'time': 730, 'lat': 600, 'lon': 1440} | {'time': 365, 'lat': 262, 'lon': 262} | 95.577469 | 12.586679 | float32 |
| 7 | single_chunk_store_lat2048_lon4096.zarr | 835.925926 | {'time': 1, 'lat': 2048, 'lon': 4096} | {'time': 1, 'lat': 2048, 'lon': 4096} | 64.000000 | 1.000000 | float64 |
| 8 | single_chunk_store_lat2896_lon5792.zarr | 1500.000000 | {'time': 1, 'lat': 2896, 'lon': 5792} | {'time': 1, 'lat': 2896, 'lon': 5792} | 127.972656 | 1.000000 | float64 |
| 9 | single_chunk_store_lat4096_lon8192.zarr | 3033.333333 | {'time': 1, 'lat': 4096, 'lon': 8192} | {'time': 1, 'lat': 4096, 'lon': 8192} | 256.000000 | 1.000000 | float64 |