NETWORK Procedure
Example 2.13 Shortest Paths of the New York Road Network
This example looks at the road networks in the state of New York (NY). The distance graph raw data are maintained at the DIMACS challenge website (Demetrescu 2010). The NY road network includes 264,346 intersections (nodes) and 733,846 roads (links). Although the input data table is not large, the computing power that you need to find all-pairs shortest paths is enormous. In addition, the storage space that you need to handle the results data can easily overwhelm the capacity of a single machine. In this example, a session of 16 machines (each with 16 cores) was configured to process this graph.
The following statements construct the links data table mylib.RoadNY from a local copy of the raw distance graph data:
filename in 'USA-road-d.NY.gr';
data mylib.RoadNY (drop=a);
infile in firstobs=8;
input a $ from $ to $ weight;
run;
The following statements find the all-pairs shortest paths of the NY road network (that have a total path weight of less than 20,000) by using a distributed algorithm. This algorithm takes advantage of all the machines and cores in your configured session.
proc network
logFreqTime = 10
logLevel = aggressive
direction = directed
links = mylib.RoadNY;
shortestPath
maxLinkWeight = 20000
outWeights = mylib.shortPathSummary
outPathsLinks = mylib.shortPathPaths;
run;
%put &_NETWORK_;
The progress of the procedure is shown in Output 2.13.1.
Output 2.13.1: PROC NETWORK Log: Shortest Paths of the NY Road Network
| NOTE: ------------------------------------------------------------------------------------------ |
| NOTE: ------------------------------------------------------------------------------------------ |
| NOTE: Running NETWORK. |
| NOTE: ------------------------------------------------------------------------------------------ |
| NOTE: ------------------------------------------------------------------------------------------ |
| NOTE: Reading the links data. |
| NOTE: Data input used 0.10 (cpu: 1.63) seconds. |
| NOTE: Building the input graph storage used 0.44 (cpu: 6.39) seconds. |
| NOTE: The number of nodes in the input graph is 264346. |
| NOTE: The number of links in the input graph is 733846. |
| NOTE: Processing shortest paths problem using 256 threads across 16 machines. |
| NOTE: Processing the shortest paths problem between 264346 source nodes and 264346 sink nodes. |
| Real |
| Algorithm Sources Complete Time |
| shortestPath 2400 0% 9.08 |
| shortestPath 7040 2% 19.51 |
| shortestPath 11520 4% 29.02 |
| shortestPath 15840 5% 39.31 |
| shortestPath 19520 7% 49.23 |
| shortestPath 23840 9% 59.29 |
| shortestPath 28480 10% 69.29 |
| shortestPath 32800 12% 79.10 |
| shortestPath 36800 13% 89.01 |
| shortestPath 41760 15% 99.53 |
| shortestPath 46080 17% 109.12 |
| shortestPath 51200 19% 119.27 |
| shortestPath 55680 21% 129.03 |
| shortestPath 59520 22% 139.89 |
| shortestPath 62800 23% 149.11 |
| shortestPath 65920 24% 159.05 |
| shortestPath 70000 26% 169.21 |
| shortestPath 73760 27% 179.24 |
| shortestPath 77415 29% 189.24 |
| shortestPath 80960 30% 199.53 |
| shortestPath 84640 32% 209.02 |
| shortestPath 88480 33% 219.86 |
| shortestPath 92640 35% 230.06 |
| shortestPath 96295 36% 240.07 |
| shortestPath 100960 38% 250.46 |
| shortestPath 104905 39% 260.44 |
| shortestPath 109440 41% 270.43 |
| shortestPath 115040 43% 280.89 |
| shortestPath 119200 45% 290.24 |
| shortestPath 123840 46% 300.10 |
| shortestPath 128320 48% 310.45 |
| shortestPath 132480 50% 320.69 |
| shortestPath 137120 51% 330.51 |
| shortestPath 141600 53% 341.26 |
| shortestPath 146400 55% 351.13 |
| shortestPath 150560 56% 361.31 |
| shortestPath 154880 58% 371.16 |
| shortestPath 159520 60% 381.07 |
| shortestPath 164320 62% 391.35 |
| shortestPath 169280 64% 401.64 |
| shortestPath 173600 65% 411.04 |
| shortestPath 178080 67% 422.23 |
| shortestPath 182880 69% 432.24 |
| shortestPath 186880 70% 442.01 |
| shortestPath 191520 72% 453.43 |
| shortestPath 196320 74% 463.60 |
| shortestPath 200640 75% 473.65 |
| shortestPath 205440 77% 483.32 |
| shortestPath 210560 79% 493.03 |
| shortestPath 216000 81% 503.41 |
| shortestPath 219200 82% 513.30 |
| shortestPath 222045 83% 523.30 |
| shortestPath 226880 85% 533.04 |
| shortestPath 230880 87% 543.20 |
| shortestPath 235200 88% 553.09 |
| shortestPath 238560 90% 563.30 |
| shortestPath 243040 91% 573.56 |
| shortestPath 245920 93% 583.08 |
| shortestPath 249280 94% 594.29 |
| shortestPath 252960 95% 604.51 |
| shortestPath 256160 96% 614.48 |
| shortestPath 260160 98% 624.75 |
| shortestPath 264186 99% 634.17 |
| shortestPath 264346 100% 634.65 |
| NOTE: Processing the shortest paths problem used 634.64 (cpu: 94495.78) seconds. |
| NOTE: The Cloud Analytic Services server processed the request in 635.918418 seconds. |
| NOTE: The data set MYLIB.SHORTPATHSUMMARY has 104263396 observations and 3 variables. |
| NOTE: The data set MYLIB.SHORTPATHPATHS has 1419295876 observations and 6 variables. |
| STATUS=OK PROBLEM_TYPE=SHORTESTPATH SOLUTION_STATUS=OK NUM_PATHS=104263396 |
| CPU_TIME=95240.33 REAL_TIME=635.92 |
Notice that the resulting output data tables, mylib.shortPathSummary and mylib.shortPathPaths, are large distributed data tables.