NETWORK Procedure
Example 2.13 Shortest Paths of the New York Road Network
This example looks at the road networks in the state of New York (NY). The distance graph raw data are maintained at the DIMACS challenge website (Demetrescu 2010). The NY road network includes 264,346 intersections (nodes) and 733,846 roads (links). Although the input data table is not large, the computing power that you need to find all-pairs shortest paths is enormous. In addition, the storage space that you need to handle the results data can easily overwhelm the capacity of a single machine. In this example, a session of 16 machines (each with 16 cores) was configured to process this graph.
The following statements construct the links data table mylib.RoadNY from a local copy of the raw distance graph data:
filename in 'USA-road-d.NY.gr';
data mylib.RoadNY (drop=a);
infile in firstobs=8;
input a $ from $ to $ weight;
run;
The following statements find the all-pairs shortest paths of the NY road network (that have a total path weight of less than 20,000) by using a distributed algorithm. This algorithm takes advantage of all the machines and cores in your configured session.
proc network
logFreqTime = 10
logLevel = aggressive
direction = directed
links = mylib.RoadNY;
shortestPath
maxLinkWeight = 20000
outWeights = mylib.shortPathSummary
outPathsLinks = mylib.shortPathPaths;
run;
%put &_NETWORK_;
The progress of the procedure is shown in Output 2.13.1.
Output 2.13.1: PROC NETWORK Log: Shortest Paths of the NY Road Network
| NOTE: ------------------------------------------------------------------------------------------ |
| NOTE: ------------------------------------------------------------------------------------------ |
| NOTE: Running NETWORK. |
| NOTE: ------------------------------------------------------------------------------------------ |
| NOTE: ------------------------------------------------------------------------------------------ |
| NOTE: Reading the links data. |
| NOTE: Data input used 0.11 (cpu: 1.83) seconds. |
| NOTE: Building the input graph storage used 0.39 (cpu: 6.73) seconds. |
| NOTE: The number of nodes in the input graph is 264346. |
| NOTE: The number of links in the input graph is 733846. |
| NOTE: Processing shortest paths problem using 256 threads across 16 machines. |
| NOTE: Processing the shortest paths problem between 264346 source nodes and 264346 sink nodes. |
| Real |
| Algorithm Sources Complete Time |
| shortestPath 2240 0% 9.33 |
| shortestPath 7040 2% 19.50 |
| shortestPath 12320 4% 30.46 |
| shortestPath 16880 6% 40.03 |
| shortestPath 20960 7% 50.74 |
| shortestPath 25600 9% 60.19 |
| shortestPath 30400 11% 70.12 |
| shortestPath 35200 13% 80.12 |
| shortestPath 39760 15% 90.05 |
| shortestPath 44320 16% 100.49 |
| shortestPath 49280 18% 110.00 |
| shortestPath 54400 20% 120.93 |
| shortestPath 58160 22% 130.03 |
| shortestPath 61795 23% 140.14 |
| shortestPath 64960 24% 150.05 |
| shortestPath 68960 26% 160.54 |
| shortestPath 72880 27% 170.55 |
| shortestPath 76480 28% 180.06 |
| shortestPath 80370 30% 190.43 |
| shortestPath 84265 31% 200.12 |
| shortestPath 87350 33% 210.20 |
| shortestPath 91680 34% 220.60 |
| shortestPath 96000 36% 230.07 |
| shortestPath 100960 38% 241.23 |
| shortestPath 105015 39% 251.34 |
| shortestPath 107945 40% 261.08 |
| shortestPath 110620 41% 271.41 |
| shortestPath 112960 42% 282.08 |
| shortestPath 115955 43% 292.08 |
| shortestPath 118480 44% 302.45 |
| shortestPath 121135 45% 312.03 |
| shortestPath 123375 46% 322.15 |
| shortestPath 126190 47% 332.15 |
| shortestPath 128465 48% 342.51 |
| shortestPath 130880 49% 352.06 |
| shortestPath 133295 50% 362.07 |
| shortestPath 136000 51% 372.24 |
| shortestPath 138560 52% 382.04 |
| shortestPath 141070 53% 392.05 |
| shortestPath 143485 54% 403.40 |
| shortestPath 146160 55% 413.33 |
| shortestPath 148735 56% 423.85 |
| shortestPath 150770 57% 433.40 |
| shortestPath 153005 57% 443.22 |
| shortestPath 155480 58% 454.25 |
| shortestPath 158475 59% 466.94 |
| shortestPath 161095 60% 476.15 |
| shortestPath 163760 61% 486.55 |
| shortestPath 166880 63% 496.10 |
| shortestPath 169600 64% 506.66 |
| shortestPath 171715 64% 516.11 |
| shortestPath 174015 65% 526.12 |
| shortestPath 176385 66% 536.15 |
| shortestPath 178640 67% 546.08 |
| shortestPath 181005 68% 557.70 |
| shortestPath 183320 69% 567.07 |
| shortestPath 185610 70% 577.09 |
| shortestPath 188395 71% 587.79 |
| shortestPath 191080 72% 597.50 |
| shortestPath 193770 73% 607.25 |
| shortestPath 196080 74% 617.18 |
| shortestPath 199290 75% 627.95 |
| shortestPath 201760 76% 637.90 |
| shortestPath 204100 77% 647.90 |
| shortestPath 206880 78% 657.67 |
| shortestPath 209840 79% 667.16 |
| shortestPath 212800 80% 678.25 |
| shortestPath 215625 81% 688.36 |
| shortestPath 217360 82% 698.36 |
| shortestPath 219200 82% 708.57 |
| shortestPath 221165 83% 718.85 |
| shortestPath 223210 84% 728.92 |
| shortestPath 225460 85% 738.93 |
| shortestPath 227505 86% 748.06 |
| shortestPath 230240 87% 758.47 |
| shortestPath 232735 88% 768.27 |
| shortestPath 234750 88% 778.04 |
| shortestPath 236970 89% 788.13 |
| shortestPath 239025 90% 798.14 |
| shortestPath 241180 91% 808.19 |
| shortestPath 243125 91% 818.80 |
| shortestPath 244920 92% 828.36 |
| shortestPath 246590 93% 838.37 |
| shortestPath 248455 93% 848.55 |
| shortestPath 250200 94% 859.35 |
| shortestPath 252020 95% 870.04 |
| shortestPath 254210 96% 880.32 |
| shortestPath 256515 97% 890.40 |
| shortestPath 258780 97% 900.05 |
| shortestPath 260955 98% 910.05 |
| shortestPath 263486 99% 920.00 |
| shortestPath 264346 100% 927.24 |
| NOTE: Processing the shortest paths problem used 927.22 (cpu: 99373.52) seconds. |
| NOTE: The Cloud Analytic Services server processed the request in 928.918645 seconds. |
| NOTE: The data set MYLIB.SHORTPATHSUMMARY has 104263396 observations and 3 variables. |
| NOTE: The data set MYLIB.SHORTPATHPATHS has 1419295876 observations and 6 variables. |
| STATUS=OK PROBLEM_TYPE=SHORTESTPATH SOLUTION_STATUS=OK NUM_PATHS=104263396 |
| CPU_TIME=100655.74 REAL_TIME=928.92 |
Notice that the resulting output data tables, mylib.shortPathSummary and mylib.shortPathPaths, are large distributed data tables.