A method to detect Lua cheats

The code I wrote for this article can be found here.

When detecting cheats in the game Garry’s Mod you have countless options. Suppose we want to detect a known Lua file (containing a cheat) that a client is executing. Among various possibilities there are the following options:

Checking for global variables (e.g. if CHEAT_CONFIG then ... end)
Checking for global functions (e.g. if isfunction(OpenCheat) then ... end)
Checking for hooks (e.g. if hook.GetTable()["Think"]["Aimbot"] then ... end)
Detour functions (like RunString()) and check the content of the arguments

Cheat developers are not stupid and localize their variables and functions in most cases:

-- cheat (executed first)
local CHEAT_CONFIG = {...}

local runstring = RunString
runstring([[
   -- more cheat code
]])

-- anticheat
-- global variable is not accessible any more
if CHEAT_CONFIG then
   BanPlayer(...)
end

-- function won't get called
local runstring = RunString
function RunString(code, ...)
   if code == "any cheat code" then
      BanPlayer(...)
   end
end

If done correctly, there are hardly any possibilities to recognize this cheat clientside with conventional methods.

In Garry’s Mod scripts are JIT compiled using LuaJIT 2.1.0-beta3. If new Lua code is executed, it is compiled before execution. However, the compilation process can be hooked. You can attach callbacks to a number of compiler events with jit.attach.

In combination with debug.getinfo, details such as the name of the function can be retrieved for each individual function that has been translated into bytecode:

local function byte_code(proto)
    debug.sethook(function(event, line)
        -- get the name fields and location fields
        local dbg = debug.getinfo(2, "nS")
        if not dbg then return end

        PrintTable(dbg)
    end, "c")
end

jit.attach(byte_code,"bc")

We execute the following code:

RunString([[
   -- useless code for demonstration
   local string_len = string.len
   local i_pairs = ipairs
   local secret = {"a", "b"}
   local new_secret = ""
   for i, v in i_pairs(secret) do
      new_secret = new_secret .. v
   end
   local len = string_len(new_secret)
]])

Every function and variable is localized. In a realistic scenario, the code would not be executed via RunString, but would be injected by an external program. However, our method would still work:

["lastlinedefined"] = 9
["short_src"] = RunString(Ex)
["source"] = @RunString(Ex)

["lastlinedefined"] = -1
["name"] = i_pairs

["lastlinedefined"] = -1
["name"] = (for generator)

["lastlinedefined"] = -1
["name"] = (for generator)

["lastlinedefined"] = -1
["name"] = (for generator)

["lastlinedefined"] = -1
["name"] = string_len
["short_src"] = [builtin:len]
["source"] = len

Capped console output after RunString was called

Now we can define a table that contains the values of the “name” key. This allows us to create a kind of signature for cheat code and assign the name of the cheat to it. If all these names are executed by a client in a sequence, we can identify the cheat:

local calls = {["i_pairs"] = {
    functions = {
        "(for generator)",
        "(for generator)",
        "(for generator)",
        "string_len"
    },
    count = 4,
    cheat_name = "Super Epic Cheat V2 Ultra Pro++"
}}

Now we adjust the function:

local function byte_code(proto)
    local current_check = {}
    local cntr = 1
    local nm = ""
    local fin = 1
    debug.sethook(function(event, line)
        -- get the name fields and location fields
        local dbg = debug.getinfo(2, "nS")
        if not dbg then return end
        local name = dbg.name
        if not name then return end
        local call = calls[name]
        -- we found the first call that might fit the cheat pattern
        if call then
            current_check = call.functions
            cntr = 1
            fin = call.count
            nm = call.cheat_name
            return
        end
        -- check if name matched the next expected function
        if current_check[cntr] == name then
            -- increment the counter to the next expected function name
            cntr = cntr + 1
            -- if we reached the end of the function list, print the cheat 
            if cntr > fin then
                debug.sethook()
                print(string.format("Detected cheat %q", nm))
            end
        else
            current_check = {}
            cntr = 1
            fin = 1
        end
    end, "c")
end

Output: Detected cheat "Super Epic Cheat V2 Ultra Pro++"

Optimization

The above code is inefficient, especially with real-time applications and the expectation of a high frame rate. A debug hook should only be active for as short a time as possible. If a player does not use this particular cheat, the hook will continue to run from the first bytecode compilation until the player exits the game. We therefore expect a match with the cheat signature in the first 30 function calls. If there is still no signature match by then, we abort the debugging:

local function byte_code(proto)
    -- ...
    local cntr_total = 0
    debug.sethook(function(event, line)
        cntr_total = cntr_total + 1
        -- only check first 30 calls
        if cntr_total > 30 then debug.sethook() end
        -- ...

This was a quick rundown of an idea I had. If you are interested, you can find my anticheat on GitHub and Steam Workshop.