Comprehensive evaluation of code understanding capabilities across multiple programming languages and tasks
Rank | Model | Accuracy |
---|---|---|
Loading Python data... |
Rank | Model | Accuracy |
---|---|---|
Loading C data... |
Rank | Model | Block 1: (1 Statement) |
Block 2: (2 Statement) |
Block 3: (3 Statement) |
---|---|---|---|---|
Loading Python data... |
Rank | Model | Block 1: (1 Statement) |
Block 2: (2 Statement) |
Block 3: (3 Statement) |
---|---|---|---|---|
Loading C data... |
Rank | Model | Accuracy |
---|---|---|
Loading Alias data... |
Rank | Model | Accuracy |
---|---|---|
Loading Branch data... |
Rank | Model | Iteration | In-Loop | Post-Loop |
---|---|---|---|---|
Loading Loop data... |
Rank | Model | Stmt Python |
Stmt C |
Block1 (Stmt 1) Py |
Block2 (Stmt 2) Py |
Block3 (Stmt 3) Py |
Block1 (Stmt 1) C |
Block2 (Stmt 2) C |
Block3 (Stmt 3) C |
Alias Acc |
Branch Pred |
Loop Iter |
In Loop |
Post Loop |
Py Output |
C Output |
Java Output |
Py Input |
C Input |
Java Input |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Loading summary data... |
Rank | Model | Python | C | Java |
---|---|---|---|---|
Loading Output data... |
Rank | Model | Python | C | Java |
---|---|---|---|---|
Loading Input data... |