Lc0 T80 Artemis Games
Moderators: Elijah, Igbo, timetraveller
-
- Forum Contributions
- Points: 107,00
- Posts: 171
- Joined: 03/12/2022, 1:38
- Status: Offline (Active 6 Hours, 4 Minutes ago)
- Medals: 1
- Topics: 27
- Reputation: 794
- Has thanked: 296 times
- Been thanked: 973 times
Lc0 T80 Artemis Games
These are from .tar.bz2 files located at https://storage.lczero.org/files/training_pgns/. Each day ends up being about 1.5 GB in size. These are Artemis games, though. I define those as games that are between 10-250 moves, end in checkmate, are judged by the Interesting Wins Search Tool to be "very interesting" and are judged by ChessBase 17 to be both "strong" and "most beautiful" (i.e. three-medal beautiful). This reduces one-and-a-half gigabytes down to a megabyte.
These are the games for yesterday (2024-04-20). They oddly come out to exactly 800. Full tags are provided, including openings and beauty.
https://pixeldrain.com/u/dWfBo9Ks
These are the games for yesterday (2024-04-20). They oddly come out to exactly 800. Full tags are provided, including openings and beauty.
https://pixeldrain.com/u/dWfBo9Ks
-
- Forum Contributions
- Points: 107,00
- Posts: 171
- Joined: 03/12/2022, 1:38
- Status: Offline (Active 6 Hours, 4 Minutes ago)
- Medals: 1
- Topics: 27
- Reputation: 794
- Has thanked: 296 times
- Been thanked: 973 times
Lc0 T80 Artemis Games
2024.04.22 (1638 games):
https://pixeldrain.com/u/wFU4pjz5
These are all Artemis games, filtered from 1.5 GB down to about 1 MB. 10-100 moves, 2300+, ending in checkmate, and which the Interesting Wins search tool rates "very interesting", and which ChessBase rates both "strong" and "most beautiful".
https://pixeldrain.com/u/wFU4pjz5
These are all Artemis games, filtered from 1.5 GB down to about 1 MB. 10-100 moves, 2300+, ending in checkmate, and which the Interesting Wins search tool rates "very interesting", and which ChessBase rates both "strong" and "most beautiful".
-
- Forum Contributions
- Points: 107,00
- Posts: 171
- Joined: 03/12/2022, 1:38
- Status: Offline (Active 6 Hours, 4 Minutes ago)
- Medals: 1
- Topics: 27
- Reputation: 794
- Has thanked: 296 times
- Been thanked: 973 times
Lc0 T80 Artemis Games
Okay. This project got off to a strange start. ChatGPT has written me a couple scripts that automate the process so well that the Interesting Wins Search Tool became a real bottleneck. So I'm combining all days from a particular month, which means I won't have April 2024 for a few more days. Here, though, is what I've been doing today.
January 2024
https://pixeldrain.com/u/cfyYL2QK
February 2024
https://pixeldrain.com/u/srnBTh8r
March 2024
https://pixeldrain.com/l/9nuoQbr8
All games end in checkmate, are between 10-100 moves, and with both players rated at least 2300. They are also considered "very interesting", "strong", and "most beautiful", according to the Interesting Wins Search Tool and ChessBase, respectively. 960 games and doubles have been removed. Includes opening tags and beauty scores.
January 2024
https://pixeldrain.com/u/cfyYL2QK
February 2024
https://pixeldrain.com/u/srnBTh8r
March 2024
https://pixeldrain.com/l/9nuoQbr8
All games end in checkmate, are between 10-100 moves, and with both players rated at least 2300. They are also considered "very interesting", "strong", and "most beautiful", according to the Interesting Wins Search Tool and ChessBase, respectively. 960 games and doubles have been removed. Includes opening tags and beauty scores.
-
- Forum Contributions
- Points: 107,00
- Posts: 171
- Joined: 03/12/2022, 1:38
- Status: Offline (Active 6 Hours, 4 Minutes ago)
- Medals: 1
- Topics: 27
- Reputation: 794
- Has thanked: 296 times
- Been thanked: 973 times
Lc0 T80 Artemis Games
Well... it's all a learning experience. I was using the wrong system to create the files, so having figured it out (after spending all day with ChatGPT) these are the Lc0 T80 test games from yesterday, filtered to include only those that are "very interesting", "strong", and "most beautiful".
2024.04.24 (3460 games)
https://pixeldrain.com/u/iNBr7Z3i
2024.04.24 (3460 games)
https://pixeldrain.com/u/iNBr7Z3i
-
- Forum Contributions
- Points: 107,00
- Posts: 171
- Joined: 03/12/2022, 1:38
- Status: Offline (Active 6 Hours, 4 Minutes ago)
- Medals: 1
- Topics: 27
- Reputation: 794
- Has thanked: 296 times
- Been thanked: 973 times
Lc0 T80 Artemis Games
Okay, there's no interest here. But the Artemis collections of Lc0 Test Games are still being made. They're being archived here:
https://github.com/ianrastall/misc-chess-projects/tree/main/lczero-test-games/t80
Hope all are well.
https://github.com/ianrastall/misc-chess-projects/tree/main/lczero-test-games/t80
Hope all are well.
-
- Forum Contributions
- Points: 107,00
- Posts: 171
- Joined: 03/12/2022, 1:38
- Status: Offline (Active 6 Hours, 4 Minutes ago)
- Medals: 1
- Topics: 27
- Reputation: 794
- Has thanked: 296 times
- Been thanked: 973 times
Lc0 T80 Artemis Games
All the games from T80 would qualify in Chessbase as strong games if they're at least ten moves, since they're already always above 2300. Only about half of them would be considered beautiful at all, and only about ten percent of those would be three medal beautiful. And out of those, only maybe half would be considered very interesting.
So these are games that are "strong", "most beautiful", and "very interesting". Each original PGN is 1.5 GB (which is too much to archive day-by-day) and each Artemis PGN is 2 MB. Approximately, but still. If you plug these into ChessBase, it will read "Top Game" on every one, and will also show three medals for a beauty score. Analysis also tends to reveal that these are exceptional games.
-
- Forum Contributions
- Points: 107,00
- Posts: 171
- Joined: 03/12/2022, 1:38
- Status: Offline (Active 6 Hours, 4 Minutes ago)
- Medals: 1
- Topics: 27
- Reputation: 794
- Has thanked: 296 times
- Been thanked: 973 times
Lc0 T80 Artemis Games
In case anyone is interested, here is the code ChatGPT wrote for me to quickly process this stuff.
The Windows PowerShell script for taking a folder of the original .tar.bz2 files and converting them to one PGN:
The Bash script (for Windows) for stripping out FENs and Lc0-specific stuff.
The Bash script (for Windows) for adding in initial headers and blank lines:
The Windows Command Line script for running pgn-extract to further clean the PGN:
After running through outside programs (the Interesting Wins Search Tool and ChessBase) the output CB file is then turned into the finished product with this Command Line script for pgn-extract:
And finally this is what's in roster.txt:
Hope all are well.
The Windows PowerShell script for taking a folder of the original .tar.bz2 files and converting them to one PGN:
Code: Select all
Get-ChildItem -Filter *.tar.bz2 | ForEach-Object {
& tar -xOjf $_ | Out-File -FilePath raw-output.pgn -Append
}
Code: Select all
awk '!/^$/ { gsub(/\{OL: 0\}.*/, ""); print }' raw-output.pgn > cleaned_output.pgn
Code: Select all
awk 'BEGIN {
header = "\n[Event \"T80\"]\n[Site \"storage.lczero.org\"]\n[Date \"2024.04.28\"]\n[Round \"?\"]\n[White \"Lc0 0.30.0\"]\n[Black \"Lc0 0.30.0\"]\n[Result \"*\"]\n[WhiteElo \"3633\"]\n[BlackElo \"3633\"]\n[EventDate \"2022.04.04\"]\n\n"
}
{
print header $0 # Print the header followed by the current line (the moves)
}' cleaned_output.pgn > updated_output.pgn
Code: Select all
pgn-extract.exe updated_output.pgn -Rroster.txt --xroster --minmoves 10 --fixresulttags --fixtagstrings --nosetuptags -D -e --plycount -w9999 -olc0-t80-filtered-240428.pgn
Code: Select all
pgn-extract.exe game1.pgn -Rroster.txt --xroster --minmoves 10 --fixresulttags --fixtagstrings --nosetuptags -D -e --plycount -w9999 -olc0-t80-artemis-240428.pgn
Code: Select all
Event
Site
Date
Round
White
Black
Result
SetUp
WhiteElo
BlackElo
ECO
Opening
Variation
EventCountry
EventDate
TimeControl
Beauty
PlyCount
-
- Inactive User
- Points: 69,00
- Posts: 8
- Joined: 09/11/2019, 19:58
- Status: Offline (Active 7 Hours, 31 Minutes ago)
- Topics: 1
- Reputation: 0
- Has thanked: 23 times
- Been thanked: 1 time
Lc0 T80 Artemis Games
So, if I'm understanding correctly, "Artemis" is your code name for the selected games.FritzUser wrote: ↑02/05/2024, 18:31All the games from T80 would qualify in Chessbase as strong games if they're at least ten moves, since they're already always above 2300. Only about half of them would be considered beautiful at all, and only about ten percent of those would be three medal beautiful. And out of those, only maybe half would be considered very interesting.
So these are games that are "strong", "most beautiful", and "very interesting". Each original PGN is 1.5 GB (which is too much to archive day-by-day) and each Artemis PGN is 2 MB. Approximately, but still. If you plug these into ChessBase, it will read "Top Game" on every one, and will also show three medals for a beauty score. Analysis also tends to reveal that these are exceptional games.
-
- Inactive User
- Points: 69,00
- Posts: 8
- Joined: 09/11/2019, 19:58
- Status: Offline (Active 7 Hours, 31 Minutes ago)
- Topics: 1
- Reputation: 0
- Has thanked: 23 times
- Been thanked: 1 time
Lc0 T80 Artemis Games
I've done command line and batch scripts, but PowerShell and Bash? No idea, I guess it will be similar. If I can get Lc0 training games in PGN format, it might be worth it.FritzUser wrote: ↑03/05/2024, 2:07 In case anyone is interested, here is the code ChatGPT wrote for me to quickly process this stuff.
The Windows PowerShell script for taking a folder of the original .tar.bz2 files and converting them to one PGN:
The Bash script (for Windows) for stripping out FENs and Lc0-specific stuff.Code: Select all
Get-ChildItem -Filter *.tar.bz2 | ForEach-Object { & tar -xOjf $_ | Out-File -FilePath raw-output.pgn -Append }
The Bash script (for Windows) for adding in initial headers and blank lines:Code: Select all
awk '!/^$/ { gsub(/\{OL: 0\}.*/, ""); print }' raw-output.pgn > cleaned_output.pgn
The Windows Command Line script for running pgn-extract to further clean the PGN:Code: Select all
awk 'BEGIN { header = "\n[Event \"T80\"]\n[Site \"storage.lczero.org\"]\n[Date \"2024.04.28\"]\n[Round \"?\"]\n[White \"Lc0 0.30.0\"]\n[Black \"Lc0 0.30.0\"]\n[Result \"*\"]\n[WhiteElo \"3633\"]\n[BlackElo \"3633\"]\n[EventDate \"2022.04.04\"]\n\n" } { print header $0 # Print the header followed by the current line (the moves) }' cleaned_output.pgn > updated_output.pgn
After running through outside programs (the Interesting Wins Search Tool and ChessBase) the output CB file is then turned into the finished product with this Command Line script for pgn-extract:Code: Select all
pgn-extract.exe updated_output.pgn -Rroster.txt --xroster --minmoves 10 --fixresulttags --fixtagstrings --nosetuptags -D -e --plycount -w9999 -olc0-t80-filtered-240428.pgn
And finally this is what's in roster.txt:Code: Select all
pgn-extract.exe game1.pgn -Rroster.txt --xroster --minmoves 10 --fixresulttags --fixtagstrings --nosetuptags -D -e --plycount -w9999 -olc0-t80-artemis-240428.pgn
Hope all are well.Code: Select all
Event Site Date Round White Black Result SetUp WhiteElo BlackElo ECO Opening Variation EventCountry EventDate TimeControl Beauty PlyCount
-
- Forum Contributions
- Points: 107,00
- Posts: 171
- Joined: 03/12/2022, 1:38
- Status: Offline (Active 6 Hours, 4 Minutes ago)
- Medals: 1
- Topics: 27
- Reputation: 794
- Has thanked: 296 times
- Been thanked: 973 times
Lc0 T80 Artemis Games
To do the Bash you have to install GitBash For Windows. With PowerShell, you enter in the same commands as on the command line, but for whatever reason have to preface a command with .\ so that instead of pgn-extract input.pgn you would write .\pgn-extract input.pgn. ChatGPT is very helpful with stuff like this.
Yes, that's a code name that I came up with. Artemis is the Greek goddess of the hunt. She's just a good example of someone who is strong, beautiful, and interesting. There have to be different levels of filtration, and they needed names so that I could create appropriate filenames. "Elegant" games are strong and beautiful (meaning any of 1-3 medals for a beauty score) and Artemis games are strong, very interesting, and most beautiful.
Yes, that's a code name that I came up with. Artemis is the Greek goddess of the hunt. She's just a good example of someone who is strong, beautiful, and interesting. There have to be different levels of filtration, and they needed names so that I could create appropriate filenames. "Elegant" games are strong and beautiful (meaning any of 1-3 medals for a beauty score) and Artemis games are strong, very interesting, and most beautiful.