Page MenuHomeSoftware Heritage

Add benchmark to test edge access time
ClosedPublic

Authored by haltode on Aug 7 2019, 7:02 PM.

Diff Detail

Repository
rDGRPH Compressed graph representation
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

I rebooted the azure VM and did the benchmark, here is the result for 1M edges:

        Command being timed: "java -Xmx220G -Xss10M -server -XX:PretenureSizeThreshold=512M -XX:MaxNewSize=4G -XX:+UseLargePages -XX:+UseTransparentHugePages -XX:+UseNUMA -XX:+UseTLAB -XX:+Re
sizeTLAB -cp swh-graph-0.0.1.jar org.softwareheritage.graph.benchmark.AccessEdge graph/compressed/all"                                                                                         
        User time (seconds): 316.50                                                                                                                                                            
        System time (seconds): 97.80                                                                                                                                                           
        Percent of CPU this job got: 16%                                                                                                                                                       
        Elapsed (wall clock) time (h:mm:ss or m:ss): 41:58.50                                                                                                                                  
        Average shared text size (kbytes): 0                                                                                                                                                   
        Average unshared data size (kbytes): 0                                                                                                                                                 
        Average stack size (kbytes): 0                                                                                                                                                         
        Average total size (kbytes): 0                                                                                                                                                         
        Maximum resident set size (kbytes): 124274776                                                                                                                                          
        Average resident set size (kbytes): 0                                                                                                                                                  
        Major (requiring I/O) page faults: 151                                                                                                                                                 
        Minor (reclaiming a frame) page faults: 314267                                                                                                                                         
        Voluntary context switches: 902158                                                                                                                                                     
        Involuntary context switches: 5357                                                                                                                                                     
        Swaps: 0                                                                                                                                                                               
        File system inputs: 223611496                                                                                                                                                          
        File system outputs: 2376                                                                                                                                                              
        Socket messages sent: 0                                                                                                                                                                
        Socket messages received: 0                                                                                                                                                            
        Signals delivered: 0                                                                                                                                                                   
        Page size (bytes): 4096                                                                                                                                                                
        Exit status: 0 

Used 1000000 random edges (results are in seconds):
min value: 2.0000000233721948E-7
max value: 0.001727816998027265
average: 1.0490578032659385E-6
standard deviation: 3.779805627909352E-6

Add reference to T1885 in commit message.

Use LazyLongIterator instead of array in NeighborsIterator

Add method to pick random nodes with specific type

This revision was not accepted when it landed; it landed in state Needs Review.Aug 8 2019, 3:03 PM
This revision was automatically updated to reflect the committed changes.