Timeseries Gap Identification
In this example we will use `nadi_csv` plugin to load CSV files. The plugin is external, refer to the installation page to install it into the NADI System.
Different from other chapters, the code in this chapters are run in the order they are given, meaning each task code blocks are not independent.
Load the network
network load_file("data/scioto/scioto.network")
network cairo.table("<Name => {NAME}", "output/scioto-net.svg")
Results:
Load timeseries data from CSV, and convert any timeseries without gaps into a complete one.
We can see the number of valid data and number of total data to see that the timeseries have gaps on them.
network csv.load_timeseries("data/scioto/scioto.csv", "date", "streamflow");
# in future version csv.load_timeseries should do this while loading
nodes do ts_complete("streamflow")
nodesmap array(ts_len("streamflow", valid=true), ts_len("streamflow"))
Results:
{
"03229610" = [7032, 38078],
"03229500" = [36616, 38078],
"03228689" = [1119, 38078],
"03228500" = [31502, 38078],
"03228300" = [13237, 38078],
"03228805" = [22495, 38078],
"03228750" = [11747, 38078],
"03227500" = [38067, 38078],
"03227107" = [3639, 38078],
"03226800" = [21306, 38078],
"03225500" = [35155, 38078],
"03223425" = [10319, 38078],
"03221646" = [3744, 38078],
"03219781" = [1234, 38078],
"03220000" = [29678, 38078],
"03221000" = [37896, 38078],
"03219500" = [33695, 38078],
"03217500" = [13604, 38078],
"03217424" = [1285, 38078]
}
None of the timeseries are complete. We can visualize the gaps using the
nodes.good = (ts_len("streamflow", valid=true) / ts_len("streamflow")) > 0.75
nodes.visual = {};
nodes.visual.nodeshape = "circle";
nodes(good).visual.nodecolor = "darkgreen";
nodes(good).visual.textcolor = "darkgreen";
network svg_ts_blocks("output/scioto-ts-gap-id.svg", "{NAME}", "streamflow", 620.0, 820.0, arr_width=500.0, bgcolor="#ffffff33")
Results:
