Lookup Table

Bribe · May 14, 2024

You can use Lua's table.pack and table.unpack to avoid the need for limiting the number of parameters. This has some uses, though I am pretty sure @Tasyen has done similar stuff, like caching the results of FourCC, that sort of thing.

Antares · May 14, 2024

If I understand you correctly, you want to use that for the __call function? Wouldn't that create a new table on each call? Since you want to use this on functions you call many times per second, that would probably overwhelm the GC. Correct me if I'm wrong.

What I could do maybe is generate the __index and __call functions for X parameters with load().

I also want to expand this by making it possible to add functions to it that use floats as parameters using linear interpolation.

Tasyen · May 14, 2024

nice library

Bribe said:
This has some uses, though I am pretty sure @Tasyen has done similar stuff, like caching the results of FourCC, that sort of thing.

I have but only for a specific table with one arg.

For FourCC I published a rejected snippet to autouse FourCC as key and maybe as value in a table.
Allowed
SpellAction['A03F'] = SpellAction['A0G2']
SpellAction['A0G2'] = function

Bribe · May 14, 2024

Antares said:
If I understand you correctly, you want to use that for the __call function? Wouldn't that create a new table on each call? Since you want to use this on functions you call many times per second, that would probably overwhelm the GC. Correct me if I'm wrong.

What I could do maybe is generate the __index and __call functions for X parameters with load().

I also want to expand this by making it possible to add functions to it that use floats as parameters using linear interpolation.

I think saving a few nanoseconds of performance is not worth it, unless we're talking about avoiding new handle creations (like replacing rects with coordinates and recycling things with MoveRect or MoveLocation, or one cached dummy unit or one cachd GetWorldBounds return). Players are already cached in frameworks like @TriggerHappy 's wc3ts and there have been countless JASS resources that do the same.

So if your concern is performance, avoiding tables, then the goal should be to hit the key areas that benefit from having a cached response, such as using overriding hooks. The Hook library does this fairly well, and last I checked doesn't even need to use things like table.pack or unpack anymore to get the job done.

Antares · May 14, 2024

Bribe said:
I think saving a few nanoseconds of performance is not worth it, unless we're talking about avoiding new handle creations (like replacing rects with coordinates and recycling things with MoveRect or MoveLocation, or one cached dummy unit or one cachd GetWorldBounds return). Players are already cached in frameworks like @TriggerHappy 's wc3ts and there have been countless JASS resources that do the same.

I can probably get rid of the argument cap with the load function. Other than removing the cap, is there an upside to what you're proposing? Just to be clear, is this what you're proposing? (cobbled together in a few seconds, don't bully)

Lua:

    table.__call = function(self, ...)
        local args = table.pack(...)
        local subTable = self
        for i = 1, #args - 1 do
            subTable = subTable[args[i]]
        end
        return subTable[args[#args]]
    end

I'm aware this probably isn't the most useful resource, but might have some uses when performance is important.

So if your concern is performance, avoiding tables, then the goal should be to hit the key areas that benefit from having a cached response, such as using overriding hooks. The Hook library does this fairly well, and last I checked doesn't even need to use things like table.pack or unpack anymore to get the job done.

Isn't the Hook library your library? Or are you talking about a different one?

Bribe · May 14, 2024

Yes, the Hook library is mine, but it got a lot of feedback from the community that made it a lot more powerful, and it's also now much faster than it used to be. I dare to say that it's as efficient as possible, but even I know better than to make such an argument. I'm sure a Nestharus kind of person could poke lots of holes in it.

Antares · May 14, 2024

I was just asking because you said "last I checked." That's an odd phrasing when you're talking about your own library.

Anyway, I just managed to get rid of the argument restriction (still have to clean it up). Did end up using pack and unpack.

Lua:

local function Test(a, b, c, d)
    return a*b*c*d
end

local function CreateIndexMethod(lookupTable, whichFunction, N)
    local iterate
    iterate = function(whichTable, i, ...)
        local args = table.pack(...)
        if i > 1 then
            setmetatable(whichTable, {__index = function(newTable, newKey)
                newTable[newKey] = {}
                iterate(newTable[newKey], i-1, newKey, table.unpack(args))
                return newTable[newKey]
            end})
        else
            setmetatable(whichTable, {__index = function(newTable, newKey)
                args[#args + 1] = newKey
                newTable[newKey] = whichFunction(table.unpack(args))
                return newTable[newKey]
            end})
        end
    end

    iterate(lookupTable, N)
    return lookupTable
end

function CreateNArgCallFunction(N)
    local code = "return function(self, "
    for i = 1, N - 1 do
        code = code .. "arg" .. i .. ", "
    end
    code = code .. "arg" .. N .. ")\nreturn self"
    for i = 1, N do
        code = code .. "[arg" .. i .. "]"
    end
    code = code .. "\nend"
    return load(code)()
end

---@param whichFunction string | function
---@param numArgs? integer
function AddLookupTable(whichFunction, numArgs)
    local isGlobalFunc = type(whichFunction) == "string"
    numArgs = numArgs or 1

    local func
    if isGlobalFunc then
        _G[whichFunction .. "Func"] = _G[whichFunction]
        func = _G[whichFunction]
    else
        func = whichFunction
    end

    local lookupTable = {}
    CreateIndexMethod(lookupTable, whichFunction, numArgs)
    getmetatable(lookupTable).__call = CreateNArgCallFunction(numArgs)

    if isGlobalFunc then
        _G[whichFunction] = lookupTable
    else
        return lookupTable
    end
end

local lookupTable = AddLookupTable(Test, 4)
print(lookupTable[2][3][4][5])
print(lookupTable(2, 3, 4, 5))

Bribe · May 14, 2024

Lua/Hook.lua at master · BribeFromTheHive/Lua

Contribute to BribeFromTheHive/Lua development by creating an account on GitHub.

github.com

Yes when I say "last I checked", it's because I haven't worked on it seriously in probably a year or more, and I have been doing a lot of coding outside of Lua since then. I think Hook is the right solution to this resource, but it has a steep learning curve.

Antares · May 15, 2024

I still don't understand what exactly you're suggesting is the shortcoming of my resource that would be solved by using Hook. The use cases aren't nearly as many as with Hook, but for what it does, it is much easier to use and it's, I think, as fast as it can possibly be. I'm sure that it can be expanded to more areas, but I don't think expanding it into areas that are already covered by your resource is the way to go.

You can use the current version for temporary lookup tables, such as this:

Lua:

    local function CalculateAngleBetweenAllUnits()
        local x = AddLookupTable(GetUnitX)
        local y = AddLookupTable(GetUnitY)

        for i = 1, #unitList do
            for j = i+1, #unitList do
                angleBetweenUnits[i][j] = math.atan(y[unitList[i]] - y[unitList[j]], x[unitList[i]] - x[unitList[j]])
                angleBetweenUnits[j][i] = -angleBetweenUnits[i][j]
            end
        end
    end

This could be improved and expanded upon. I could also see a feature where you can set a lifetime for a value, so that you can use it on variable values, but if a thousand functions all request the value at the same time, only the first call has to calculate it.

Tasyen · May 15, 2024

I did a small performance test. this boosts FourCC in Reforged quite much when requesting the same string. in v1.31.1 not so much, but still it became faster.

Lua:

oldTime = os.clock()
for index = 1, 1000000 do
    FourCC'AHbz'
end
print(os.clock()-oldTime)


print"replace"
AddLookupTable("FourCC")

-- V1.31.1
--> 0.111
-- replace
--> 0.042

-- Reforged
--> 0.545
-- replace
--> 0.04

Antares · May 15, 2024

Interesting, FourCC (or natives in general) are faster pre-Reforged? I didn't know that. What did they mess up there again?

Tasyen · May 15, 2024

Antares said:
Interesting, FourCC (or natives in general) are faster pre-Reforged? I didn't know that. What did they mess up there again?

In V1.31 FourCC breaks with Save&Load therefore they swaped the way it operates.
It was something with string.unpack in V1.31. In Reforged it does some string.byte math.

Antares · May 15, 2024

Back when they still cared about fixing those types of bugs. :pcry:

These are my results:
FourCC naked: 400ns
FourCC with Hook: 70ns
FourCC with Lookup (function call): 35ns
FourCC with Lookup (table lookup): 10ns

Bribe · May 16, 2024

Hi, sorry to take so long to get back to you.

For this:

Lua:

    function AddLookupTable(functionName)
        _G[functionName .. "Func"] = _G[functionName]
        local table = {}
        table.__index = function(self, key) self[key] = _G[functionName](key) return self[key] end
        table.__call = function(self, arg) return self[arg] end
        _G[functionName] = table
    end

Hook makes more sense in terms of allowing the user to be able to call the original native.

The simpler approach is just to avoid a table altogether (let's forget the exposing of the original native just for the sake of simplicity):

Lua:

    function AddLookupTable(functionName)
        local oldFunc = _G[functionName]
        local t = {}
        _G[functionName] = function(key)
            local result = t[key]
            if result == nil then
                result = oldFunc(key)
                t[key] = result
            end
            return result
        end
    end

This will assuredly be faster, more maintainable and more memory efficient.

I'll come back to the other parts of the system a bit later.

Antares · May 16, 2024

Bribe said:
Hook makes more sense in terms of allowing the user to be able to call the original native.

You don't have to overwrite the native if you don't want to. Thinking about it, the feature of replacing the native if you call AddLookupTable with a string argument makes little sense, because if the user really wants to replace the native, they can just do:

Lua:

FourCC = AddLookupTable(FourCC)

This will assuredly be faster, more maintainable and more memory efficient.

Your version comes in at 28ns, so it's slightly faster than my version as a function call, but slower than a direct table lookup (no surprise here). More memory efficient... both versions store one table entry per different value unless I'm missing something.

You know, I could just change it so you have the option to return a table or replace the original function with your version (and remove the __call method from the table). Then you have the best of both worlds.

Bribe · May 16, 2024

That's a good point about the table lookup, I hadn't thought about that.

Bribe · May 17, 2024

So I tried testing the existing AddLookupTable function, and it didn't work because it's not assigned to a metatable, so kept getting 'attempt to call table value' errors. I ended up with the following successful test scenario:

Lua:

function bob(x)
    print 'only once'
    return 'always'
end

    function AddLookupTable(functionName)
        local old = _G[functionName]
        local table = {}
        _G[functionName] = setmetatable(table, table)
        table.__index = function(self, key)
            local result = old(key)
            self[key] = result
            return result
        end
        table.__call = function(self, arg) return self[arg] end
    end

-- bob(1)

AddLookupTable('bob')

print(bob(1))
print(bob(1))

Antares · May 17, 2024

Oh shoot, my dog must have ate the setmetatable line. I'll fix it. (I was testing with the extended version the entire time, didn't catch it).

Bribe said:
That's a good point about the table lookup, I hadn't thought about that.

The idea is that you can use the FourCC.hfoo syntax for your own coding, but it doesn't break other resources that use the FourCC"hfoo" syntax, while also making those faster.

Antares · Saturday at 9:08 AM

Update:

Added the ability to create temporary lookup tables with TempLookup. You can do something like this:
Lua:
```
    local x, y = TempLookup(GetUnitX), TempLookup(GetUnitY)
    local caster = GetSpellAbilityUnit()
    local target = GetSpellTargetUnit()
    local dx = x[caster] - x[target]
    local dy = y[caster] - y[target]
```
The x and y variables can be used any number of times in the current thread, then they get nilled. Temporary lookup tables are very performant and quite convenient to use in longer functions with a lot of accesses.
Added the ability to define a setter function for a function that gets added to a lookup table. A hook will be created with the setter function to rewrite the entry in the lookup table to the new value. The setter function must take N + 1 arguments with the same argument order as the getter function plus the last argument for the new value.
Added __mode = "k" to the metatable of the last table in the table chain.
Added some more natives to the FastNatives library.

Lookup Table

Lookup Table (Binary)

Similar threads