Comprehensive evaluation of code understanding capabilities across multiple programming languages and tasks
| Rank | Model | Accuracy |
|---|---|---|
| Loading Python data... | ||
| Rank | Model | Accuracy |
|---|---|---|
| Loading C data... | ||
| Rank | Model | Block 1: (1 Statement) |
Block 2: (2 Statement) |
Block 3: (3 Statement) |
|---|---|---|---|---|
| Loading Python data... | ||||
| Rank | Model | Block 1: (1 Statement) |
Block 2: (2 Statement) |
Block 3: (3 Statement) |
|---|---|---|---|---|
| Loading C data... | ||||
| Rank | Model | Accuracy |
|---|---|---|
| Loading Alias data... | ||
| Rank | Model | Accuracy |
|---|---|---|
| Loading Branch data... | ||
| Rank | Model | Iteration | In-Loop | Post-Loop |
|---|---|---|---|---|
| Loading Loop data... | ||||
| Rank | Model | Stmt Python |
Stmt C |
Block1 (Stmt 1) Py |
Block2 (Stmt 2) Py |
Block3 (Stmt 3) Py |
Block1 (Stmt 1) C |
Block2 (Stmt 2) C |
Block3 (Stmt 3) C |
Alias Acc |
Branch Pred |
Loop Iter |
In Loop |
Post Loop |
Py Output |
C Output |
Java Output |
Py Input |
C Input |
Java Input |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Loading summary data... | ||||||||||||||||||||
| Rank | Model | Python | C | Java |
|---|---|---|---|---|
| Loading Output data... | ||||
| Rank | Model | Python | C | Java |
|---|---|---|---|---|
| Loading Input data... | ||||