tool to drive an engine

[deeds]

Hi folks, i'm searching a tool to automate this :

we give it the position, the move and a depth => the tool runs an engine => configure the position (position fen xxx) => analyses the move (go depth xx moves xxxx) => quit the engine => display the last info string into the console.

Thank you by advance

[Nemesis]

Hi folks, i'm searching a tool to automate this :

we give it the position, the move and a depth => the tool runs an engine => configure the position (position fen xxx) => analyses the move (go depth xx moves xxxx) => quit the engine => display the last info string into the console.

Thank you by advance

Not sure about this but 'Ferdy' at TalkChess coded many such batch files - you may want to check at TalkChess.

[Nemesis]

Have a look at this .. dated but useful nevertheless

https://github.com/zeFresk/deep-position-analysis

[deeds]

Thank you ! All this stuff helped me to better understand things. Now, i can code my own tool. My goal is to use any engine in a cluster even if they don't handle an internal MPI system.

I plan to call this by something like :
mpiexec.exe -host computer_name -dir tool_directory tool.exe [position] [move] [depth] where tool.exe will use an engine to analyse the move from the position until the depth and display only the last string with score, pv, speed, etc.

[Nemesis]

Thank you ! All this stuff helped me to better understand things. Now, i can code my own tool. My goal is to use any engine in a cluster even if they don't handle an internal MPI system.

I plan to call this by something like :
mpiexec.exe -host computer_name -dir tool_directory tool.exe [position] [move] [depth] where tool.exe will use an engine to analyse the move from the position until the depth and display only the last string with score, pv, speed, etc.

Funny you mentioned this as a few years ago I asked the very same question on the now defunct Rybka Forum - no reasonable answer forthcoming so another poster suggested a 'solution' I use now with a GUI - works for me and I'm satisfied with the analysis such as it is.

The best to you and need I say a copy of your binary would be much appreciated !!

[deeds]

Funny you mentioned this as a few years ago I asked the very same question on the now defunct Rybka Forum - no reasonable answer forthcoming...

I suppose this problem needs 2 tools when the engines don't already handle a MPI system and when all the hosts can't run the same release of the engines (bmi2, popcount, vnni).

One intermediate tool to drive the engine and display only the needed info strings. This tool can use any engine with optimized settings for each host. I just coded it.

One final tool to run the MPI commands with the different positions/moves, sum the info strings, rank the bestmoves, etc.

[deeds]

at the moment, the intermediate tool looks like this :

Code: Select all

Module modMain
    Private fileINI As String
    Private enginePath As String
    Private fen As String
    Private move As String
    Private depth As Integer
    Private engineProcess As New System.Diagnostics.Process()
    Private engineInput As System.IO.StreamWriter
    Private engineOutput As String

    Sub Main()
        Dim myString As String, tabString() As String, i As Integer
        Dim arguments As String, tabArguments() As String

        fileINI = My.Computer.Name & ".ini" 'allow to use different engines (bmi2, popcount, vnni)
        arguments = Command()

        If arguments = "" Then
            Console.WriteLine("a good command is : clustered_engine.exe [fen], [move], [depth]")
        ElseIf Not My.Computer.FileSystem.FileExists(fileINI) Then
            Console.WriteLine("the " & fileINI & "file wasn't found")
        Else
            tabArguments = Split(arguments, ", ") '[fen], [move], [depth]
            fen = tabArguments(0)
            move = tabArguments(1)
            depth = Int(tabArguments(2))

            'get path and commands for engine
            myString = My.Computer.FileSystem.ReadAllText(fileINI)
            tabString = Split(myString, vbCrLf)
            enginePath = tabString(0)

            'load engine
            loadEngine(enginePath)

            'configure engine
            For i = 1 To UBound(tabString)
                If tabString(i) <> "" Then
                    engineInput.WriteLine(tabString(i))
                End If
            Next

            'prepare engine
            engineInput.WriteLine("isready")
            While InStr(engineOutput, "readyok", CompareMethod.Text) = 0
                Threading.Thread.Sleep(10)
            End While
            engineOutput = ""

            'update window
            myString = "go depth " & depth & " searchmoves " & move
            Console.Title = fen & " => " & myString

            'set position and search move
            engineInput.WriteLine("fen fen " & fen)
            engineInput.WriteLine(myString)

            'waiting
            myString = ""
            While InStr(engineOutput, "bestmove", CompareMethod.Text) = 0
                Threading.Thread.Sleep(10)
            End While

            'get last info string
            myString = ""
            tabString = Split(engineOutput, vbCrLf)
            For i = UBound(tabString) To 0 Step -1
                If InStr(tabString(i), " depth " & depth, CompareMethod.Text) > 0 _
                And InStr(tabString(i), " pv " & move, CompareMethod.Text) > 0 Then
                    If myString = "" Then
                        myString = tabString(i)
                        Exit For
                    End If
                End If
            Next

            'update window
            Console.WriteLine("#" & Format(engineProcess.Id, "00000") & " : " & myString)

            discardEngine()
        End If
    End Sub

    Public Sub loadEngine(enginePath As String)
        engineOutput = ""

        'capture and redirection
        engineProcess.StartInfo.UseShellExecute = False
        engineProcess.StartInfo.RedirectStandardOutput = True
        AddHandler engineProcess.OutputDataReceived, AddressOf evenement
        engineProcess.StartInfo.RedirectStandardInput = True
        engineProcess.StartInfo.RedirectStandardError = False
        engineProcess.StartInfo.CreateNoWindow = True

        'absolute or relative engine's path
        If My.Computer.FileSystem.DirectoryExists(My.Computer.FileSystem.GetParentPath(enginePath)) Then
            engineProcess.StartInfo.WorkingDirectory = My.Computer.FileSystem.GetParentPath(enginePath)
        Else
            engineProcess.StartInfo.WorkingDirectory = My.Application.Info.DirectoryPath
        End If
        engineProcess.StartInfo.FileName = enginePath

        'start engine
        engineProcess.Start()
        engineProcess.PriorityClass = ProcessPriorityClass.Idle

        'uci mode
        engineInput = engineProcess.StandardInput

        engineInput.WriteLine("uci")
        engineProcess.BeginOutputReadLine()

        'waiting
        While InStr(engineOutput, "uciok", CompareMethod.Text) = 0
            Threading.Thread.Sleep(10)
        End While
        engineOutput = ""
    End Sub

    Private Sub evenement(sendingProcess As Object, myData As DataReceivedEventArgs)
        'append new data from engine's output
        If InStr(engineOutput, myData.Data) = 0 Then
            engineOutput = engineOutput & myData.Data & vbCrLf
        End If
    End Sub

    Private Sub discardEngine()
        'terminate capture and redirection
        engineInput.Close()
        engineProcess.CancelOutputRead()
        engineProcess.Close()

        'cleaning
        engineInput = Nothing
        engineProcess = Nothing
        engineProcess = Nothing
    End Sub
End Module

[Nemesis]

at the moment, the intermediate tool looks like this :

Code: Select all

Module modMain
    Private fileINI As String
    Private enginePath As String
    Private fen As String
    Private move As String
    Private depth As Integer
    Private engineProcess As New System.Diagnostics.Process()
    Private engineInput As System.IO.StreamWriter
    Private engineOutput As String

    Sub Main()
        Dim myString As String, tabString() As String, i As Integer
        Dim arguments As String, tabArguments() As String

        fileINI = My.Computer.Name & ".ini" 'allow to use different engines (bmi2, popcount, vnni)
        arguments = Command()

        If arguments = "" Then
            Console.WriteLine("a good command is : clustered_engine.exe [fen], [move], [depth]")
        ElseIf Not My.Computer.FileSystem.FileExists(fileINI) Then
            Console.WriteLine("the " & fileINI & "file wasn't found")
        Else
            tabArguments = Split(arguments, ", ") '[fen], [move], [depth]
            fen = tabArguments(0)
            move = tabArguments(1)
            depth = Int(tabArguments(2))

            'get path and commands for engine
            myString = My.Computer.FileSystem.ReadAllText(fileINI)
            tabString = Split(myString, vbCrLf)
            enginePath = tabString(0)

            'load engine
            loadEngine(enginePath)

            'configure engine
            For i = 1 To UBound(tabString)
                If tabString(i) <> "" Then
                    engineInput.WriteLine(tabString(i))
                End If
            Next

            'prepare engine
            engineInput.WriteLine("isready")
            While InStr(engineOutput, "readyok", CompareMethod.Text) = 0
                Threading.Thread.Sleep(10)
            End While
            engineOutput = ""

            'update window
            myString = "go depth " & depth & " searchmoves " & move
            Console.Title = fen & " => " & myString

            'set position and search move
            engineInput.WriteLine("fen fen " & fen)
            engineInput.WriteLine(myString)

            'waiting
            myString = ""
            While InStr(engineOutput, "bestmove", CompareMethod.Text) = 0
                Threading.Thread.Sleep(10)
            End While

            'get last info string
            myString = ""
            tabString = Split(engineOutput, vbCrLf)
            For i = UBound(tabString) To 0 Step -1
                If InStr(tabString(i), " depth " & depth, CompareMethod.Text) > 0 _
                And InStr(tabString(i), " pv " & move, CompareMethod.Text) > 0 Then
                    If myString = "" Then
                        myString = tabString(i)
                        Exit For
                    End If
                End If
            Next

            'update window
            Console.WriteLine("#" & Format(engineProcess.Id, "00000") & " : " & myString)

            discardEngine()
        End If
    End Sub

    Public Sub loadEngine(enginePath As String)
        engineOutput = ""

        'capture and redirection
        engineProcess.StartInfo.UseShellExecute = False
        engineProcess.StartInfo.RedirectStandardOutput = True
        AddHandler engineProcess.OutputDataReceived, AddressOf evenement
        engineProcess.StartInfo.RedirectStandardInput = True
        engineProcess.StartInfo.RedirectStandardError = False
        engineProcess.StartInfo.CreateNoWindow = True

        'absolute or relative engine's path
        If My.Computer.FileSystem.DirectoryExists(My.Computer.FileSystem.GetParentPath(enginePath)) Then
            engineProcess.StartInfo.WorkingDirectory = My.Computer.FileSystem.GetParentPath(enginePath)
        Else
            engineProcess.StartInfo.WorkingDirectory = My.Application.Info.DirectoryPath
        End If
        engineProcess.StartInfo.FileName = enginePath

        'start engine
        engineProcess.Start()
        engineProcess.PriorityClass = ProcessPriorityClass.Idle

        'uci mode
        engineInput = engineProcess.StandardInput

        engineInput.WriteLine("uci")
        engineProcess.BeginOutputReadLine()

        'waiting
        While InStr(engineOutput, "uciok", CompareMethod.Text) = 0
            Threading.Thread.Sleep(10)
        End While
        engineOutput = ""
    End Sub

    Private Sub evenement(sendingProcess As Object, myData As DataReceivedEventArgs)
        'append new data from engine's output
        If InStr(engineOutput, myData.Data) = 0 Then
            engineOutput = engineOutput & myData.Data & vbCrLf
        End If
    End Sub

    Private Sub discardEngine()
        'terminate capture and redirection
        engineInput.Close()
        engineProcess.CancelOutputRead()
        engineProcess.Close()

        'cleaning
        engineInput = Nothing
        engineProcess = Nothing
        engineProcess = Nothing
    End Sub
End Module

Deeds .. if the coding achieves this I'm a happy camper ..

we give it the position, the move and a depth => the tool runs an engine => configure the position (position fen xxx) => analyses the move (go depth xx moves xxxx) => quit the engine => display the last info string into the console.

[deeds]

Yes it does that, more precisely the main command is :
go depth xx searchmoves xxxx

At the moment i'm coding the logic of the second tool...

[deeds]

The second tool looks like this :





In progress...

[Nemesis]

The second tool looks like this :





In progress...

Looking good deeds ..

[deeds]

clustered_engine (intermediate tool)
cluster_manager (second tool) coming soon...

[deeds]

And the second tool is available here.

[deeds]

Looking good deeds ..

Some engines seem incompatible with the clustered_engine tool :
- rybka 4.1 (don't handle the go depth xx searchmoves xxxx commands)
- houdini 6.03 (number of activations is limited, don't show info strings before D15)
- scorpio 3 (don't quit after analysis)

Some engines can't be used as local engine with the cluster_manager tool :
- asmfish (no command to convert moves suite into fen)
- scorpio 3 (don't quit after multipv command)

Komodo and Dragon dislike the go depth xx commands. Often they even don't want to show the last info string without upperbound or lowerbound tags.

[Nemesis]

Looking good deeds ..

Some engines seem incompatible with the clustered_engine tool :
- rybka 4.1 (don't handle the go depth xx searchmoves xxxx commands)
- houdini 6.03 (number of activations is limited, don't show info strings before D15)
- scorpio 3 (don't quit after analysis)

Some engines can't be used as local engine with the cluster_manager tool :
- asmfish (no command to convert moves suite into fen)
- scorpio 3 (don't quit after multipv command)

Komodo and Dragon dislike the go depth xx commands. Often they even don't want to show the last info string without upperbound or lowerbound tags.

Terminal instruction you gave worked perfectly but when I changed the FEN it stopped working.
Will give more details Monday or Tuesday - got tied up with other work this weekend.