Category: Tutorials

A significant hurdle to testing code, and managing a game in Roblox is the tightly coupled nature of the DataStore. A good idea for dealing with the added complexity is abstracting away the idea of storing information to a swapable or customizable memory backing system.

Unifying Read/Write to a DataStore

The DataStore for Roblox is really just a glorified table that is written to a database. More than likely this is backed by Redis, information that is not really important here.

When you think about data you think about 4 essential operations: Loading, Reading, Writing and Saving.

We can create a module that has a swapable interface, and then in our code use that. During testing we can then back all of the game state with an in-memory database, and when it’s time to deploy the game, we can swap out the Loading and Saving functions to actually use the real datastore.

Logging Module

We should always connect things up with a logger, though in this example we don’t use it yet, it’s still good practice. Here’s my generic logging module:

local m = {}
      m.Level = 3 -- all messages, include info
      m.id ="Logger".. tostring(math.random(1000,9999))
      math.randomseed(math.random(9999999))
local function printable(msg)
    return {
        Print=function()
            print(msg)
        end,
        ToString=function()
            return msg
        end     
    }
end
local function table_print(tt, indent, done)
    if type(tt) == "string" then
        return tt
    end
    if type(tt) == "number" then
        return tostring(tt)
    end
    done = done or {}
    indent = indent or 0
    if type(tt) == "table" then
        local sb = {}
        for key, value in pairs (tt) do
            table.insert(sb, string.rep (" ", indent)) -- indent it
            if type (value) == "table" and not done [value] then
                done [value] = true
                table.insert(sb, "{[" .. key .. "] = ");
                table.insert(sb, table_print (value, indent + 2, done))
                table.insert(sb, string.rep (" ", indent)) -- indent it
                table.insert(sb, "}");
            elseif "number" == type(key) then
                table.insert(sb, string.format("\"%s\"\n", tostring(value)))
            else
                table.insert(sb, string.format("%s = \"%s\"\n", tostring (key), tostring(value)))
            end
        end
        return table.concat(sb)
    else
        return tostring(tt) .. "\n"
    end
end
function m.Info(msg)
    if m.Level < 3 then
        return
    end
    return printable("[Info]: " .. table_print(msg))
end
function m.Warn(msg)
    if m.Level < 2 then
        return
    end
    return printable("[Warn]: " .. table_print(msg))
end
function m.Error(msg)
    if m.Level < 1 then
        return
    end
    print("m.Error: " .. msg)
    return printable("[Error]: " .. table_print(msg))
end
m.Printable = printable
m.FormatValue = table_print
m.CloneInto = function (id,loc)
    for name,func in pairs(m) do
        if type(func) == "function" then
            loc[name] = function (msgx)
                local tmp = m.FormatValue(msgx)
                tmp = id .. " " .. tmp
                return m[name](tmp)
            end
        else
            loc[name] = func
        end
    end 
    return loc
end
return m

Notice that the Logger functions Info, Warn and Error return an object with a Print and a ToString function. This is a handy convention for when you want to collect up output and consolidate it. You would call it directly like so: logger.Info("Something to note").Print()

This is really just for forward compatibility, it serves no purpose beyond that.

The MemDB module

We only need a dead simple Reader/Writer for the memory backing, with blank Save and Load functions.

We want to be able to store and access tables, and they might be non-trivial in complexity. For instance, perhaps we'll want to use Memory backing for some things, and only save or load certain bits of the table to the DataStore.

memdb.Write = function (key,value)
    local tbl = memdb.DB
    if string.find(key,"/",1,true) then
        local parts = split(key,"/")
        local c_k = nil
        for i = 1,(#parts - 1) do
            c_k = parts[i]
            if not tbl[c_k] then
                tbl[c_k] = {}
            end
            tbl = tbl[c_k]
        end 
        key = parts[#parts]
    end
    tbl[key] = value
end

Here we take advantage of lua's natural representation of tables as references. This works because each time we assign tbl, we get a reference and not a new variable, so we can drill down recursively into a table, using the '/' as a path separator. Essentially, we treat a piece of data as a "Path", so if you wanted to set a flag on a player, that lets your game know the player has the ability to jump higher, or run faster, you might save that information in a path like so: data.Write("Players/" .. player.UserId .. "/JumpPower", 200)

The Read function is almost identical:

memdb.Read = function (key)
    local tbl = memdb.DB
    if string.find(key,"/",1,true) then
        local parts = split(key,"/")
        local c_k = nil
        for i = 1,(#parts - 1) do
            c_k = parts[i]
            if not tbl[c_k] then
                tbl[c_k] = {}
            end
            tbl = tbl[c_k]
        end 
        key = parts[#parts]
    end
    return tbl[key]
end

If tbl[key] is not set, it will be nil, and it's a good idea to use nil sparringly, because it's a good marker for something like key missing, or key not set.

The full code

local function split(str, delim)
    local result,pat,lastPos = {},"(.-)" .. delim .. "()",1
    for part, pos in string.gmatch(str, pat) do
        table.insert(result, part); lastPos = pos
    end
    table.insert(result, string.sub(str, lastPos))
    return result
end
local memdb = {}
local logger = {}
memdb.DB = {}
memdb.Read = function (key)
    local tbl = memdb.DB
    if string.find(key,"/",1,true) then
        local parts = split(key,"/")
        local c_k = nil
        for i = 1,(#parts - 1) do
            c_k = parts[i]
            if not tbl[c_k] then
                tbl[c_k] = {}
            end
            tbl = tbl[c_k]
        end 
        key = parts[#parts]
    end
    return tbl[key]
end
memdb.Write = function (key,value)
    local tbl = memdb.DB
    if string.find(key,"/",1,true) then
        local parts = split(key,"/")
        local c_k = nil
        for i = 1,(#parts - 1) do
            c_k = parts[i]
            if not tbl[c_k] then
                tbl[c_k] = {}
            end
            tbl = tbl[c_k]
        end 
        key = parts[#parts]
    end
    tbl[key] = value
end
memdb.Save = function () 
    return true,nil
end
memdb.Load = function (data)
    memdb.DB = data
end
memdb.Install = function(log)
    logger = log.CloneInto("memdb",logger)
    memdb.Load({})
end
return memdb

Simple test rig

Now let's pull it all together and write a simple unit test for the two use cases:

package.path = './?.lua;' .. package.path
local data = require("memdb")
local logger = require("logger")
data.Install(logger)
data.Write("testKey","Hello from the testKey")
data.Write("this/contains/one","Hello World")
print(data.Read("this/contains/one"))
print(data.Read("testKey"))
logger.Info(data.DB).Print()

Which outputs:

D:\Projects\Roblox>lua test.lua
Hello World
Hello from the testKey
[Info]: testKey = "Hello from the testKey"
{[this] =   {[contains] =     one = "Hello World"
  }}
  1. Design your plugins/modules around Inputs and Outputs
  2. The functions receiving Inputs (and returning Outputs) are your Interface
  3. Obey the Separation of Concerns
  4. Use Dependency Injection of other modules with well defined Interfaces
  5. Never do more than you have to
  6. Never settle for 1 plugin when 2 will be cleaner

Basics of Module/Plugin Design

A module is a black box that has an input and an output (or many) and it produces a result. The user of the box should not really care how it does what it does. Designing a good module is about being very precise on what inputs you’ll get, and what outputs are required.

A module requires some set of functions for receiving inputs (called the interface) and at least one function for installation or attaching. Think of the attach function as a kind of constructor.

local BlackBox = {}
local function greet(name)
    return "Hello " .. name
end 
BlackBox.Install = function(custom_greet)
    if not custom_greet then
        BlackBox.Greet = greet
    else
        BlackBox.Greet = custom_greet
    end
end
return BlackBox

And here is how you would use the plugin:

local blackBox = require(game.ReplicatedStorage.Modules:WaitForChild("BlackBox"))
blackBox.Install()
-- OR
blackBox.Install(function(name)
    return "Hola " .. name
end)
local greeting = blackBox.Greet(player.Name)

This illustrates the basic design of a black box, or a Module. Notice how we don’t print inside of the module. Never do anything you don’t need to do inside the black box. A Module should exist for outputting messages, like logging, errors, and so on. If a Module needs to print anything, it should always accept a Printer. A Printer or Logger should have a standard interface like Log, Info, Warn, Error, Print etc. We call this Dependency Injection. Whenever your module depends on certain functionality that it really shouldn’t provide, then we need to inject that functionality into the plugin.

Separating Concerns

What should your module know about and do? The answer is: AS LITTLE AS POSSIBLE. Programmers on the go, especially new ones, often make the mistake of combining two or three modules into one big monolith that knows too much and goes too far. One example I’ve seen is an in-game currency manager that knows about and delivers assets to players.

This should really be two plugins, if you think about it, your wallet doesn’t care what you buy, or where you buy it, only that you have enough money or not.

This is a continuation of the idea that Plugins or Modules really shouldn’t be printing things, a Printer or Logger should do that, and the facility should be provided to the Module. A wallet doesn’t decide whether the money you spend is for beer or condoms or pet care magazines.

Here is some untested pseudo-code for a wallet module:

-- WalletModule
local Wallet = {}
local balance = 0
local last_error = nil
Logger = {}
Logger.Log = function (msg) return nil end
local function setBalance(amnt)
    balance = amnt
    return true
end
local function getBalance(amnt)
    return balance
end
function canDebit(amnt)
    if balance >= amnt then
        return true
    else
        return false
    end
end
local function credit(amnt)
    balance = balance + amnt
    return true
end
local function debit(amnt)
    if canDebit(amnt) then
        balance = balance - amnt
        return true
    else
        last_error = "Insufficient Balance"
        return false
    end
end
local function getLastError()
    return last_error
end
Wallet.Install = function(logger,save_func)
    Logger = logger
    Wallet.GetLastError = getLastError
    Wallet.CanDebit = canDebit
    Wallet.Debit = debit
    Wallet.Credit = credit
    Wallet.SetBalance = setBalance
    Wallet.GetBalance = getBalance
    if save_func then
        Wallet.Save = save_func
    else
        Wallet.Save = function()
            Logger.Log("Save function not implemented")
        end
    end
end
return Wallet

-- LoggerModule
local Logger = {}

Logger.Log = function(msg)
    print(msg)
end
return Logger

-- Main Server Script, or a Player section of
-- the script
local wallet = require(...)
local logger = require(...)
wallet.Install(logger,function() 
    -- Do something with DataStore here.
end)

I don’t want to get too caught up in the implementation details, suffice it to say, you only want your Modules to do one or two things, and do them well and bug free.

One thing to remember about Lua is that it is NOT Object Oriented, it is a functional programming language. So you should be thinking in terms of inputs and outputs. Now there is a kind of semi-global state to the Module in the form of balance, and the functions have side effects, but only within the module. Try as best as you can to never let a black box have side effects to the global space. If you must have some kind of side effect, make sure it is explicitly set by a function that obviously has side effects, or is marked as having them. So when tits become upwardly inclined, which they will, you can easily find the site of the side effect and make sure it’s not the cause of your sorrows (it usually will be).

Naming and Interfaces

There are many flame wars about function and variable naming, spacing, and interface design. Screw that. Just do it this way, and move on.

Local Variables/Internal Module State

Use snake_case, but spell the words out fully. No abbreviations like msg, or _t etc.

local shekel_balance = 0
local floren_balance = 0

Local/Internal Functions

Use camelCase as well.

local function canDebit()
    return true
end

Function Arguments

Use snake_case. But never use the whole word, make them short but meaningful. _t is a throwback from C, where custom types were often appended with an _t, like int_t or player_t.

local function canDebit(curr_t,amnt)
    if curr_t == "Shekel" then
        if shekel_balance >= amnt then return true end
    elseif curr_t == "Floren" then
        if floren_balance >= amnt then return true end
    else
        last_error_message = "Unknown Currency " .. curr_t
    end
end

Public Interface Functions

Use a CamelCase, with each word capitalized.

Module.CanDebit = canDebit

Getters and Setters

Make sure that you declare all “private” variables as local, and all private functions as local, assign them later to public facing functions.

Use Getters and Setters to allow outside users to manage or affect data. Don’t be stingy either. C++ programmers can really be a headache because they take the Public/Private thing way too far and hamstring their modules. There should be a valid way to change EVERYTHING inside a module. You have to grant an interface for more experienced programmers, otherwise they’ll just hack your plugin up. What’s the point?

I can’t tell you how many times I’ve just re-written or forked and modified a library because the author got cute with public/private for no good reason.

The issue with Common Lisp for imperative language programmers is that Lisp has no structure around which to orient oneself. Here I mean structure in the sense of an imperative programming language. There are a lot of symbols in imperative languages that are used to orient you, like guide rails. To an imperative programmer, Lisp leaves them awash in a sea of parentheses, they don’t see the structure that is in fact there.

(defun hello-world () 
    (print "Hello World"))
(hello-world)

Does not seem so scary to me, in fact, it looks rather like:

function hello_world()
    print("Hello World")
end
hello_world()

Our issue, at first is that there is no presence of end as a signal that definition is complete. In fact, the signal that definition is complete in Lisp is “)” which is identical to the signal that definition is beginning, or that formal arguments are being defined, or that a new “block” is being created, and ended. All of these are signaled with the same lexical symbol, which is to say that are almost not being signaled at all!

With such a trivial definition it’s easy to say that Lisp is easy, but as programs grow, our issue, the lack of signals, only grows with it.

Another issue is that I have programmed my own code highlighter. My code highlighter uses a consistent color for all “keywords.” But Lisp doesn’t really have keywords in a technical sense, because all keywords are really just more kinds of functions. This can easily become confusing. But of course, for all practical purposes, defun is a keyword. In fact, it’s possible for us to say really that any normally inbuilt function in Lisp is a keyword. Why not? Who is going to stop us?

(dotimes (i 10)
    (print i))

I think we can figure out what this does. But dotimes is a Macro! Not a keyword! Who cares? It really is, for our purposes the same difference. In fact, for almost all cases of trivial programs, the existence of macros is irrelevant.

(loop for i from 1 to 10 do (print i))

Which is a particularly lucid way to loop.

(defun ++ (x)
    (+ x 1))
(++ 1) 

I wonder what the above means? But then that indicates that even operators are functions in lisp, and we can define new ones on the fly! That is both good (after all, c++ has operator overloading) but that also means that we cannot orient ourselves by some fixed point in Lisp because anything (keywords, operators) could be (and sometimes are) redefiend! How will we know what anything means!

(defun greeter (person)
  (lambda () (print (format nil "Hello ~a" person))))
(funcall (greeter "Jolie Rouge"))

That looks a bit confusing! But is it really so different from:

function greeter(person)
    return function ()
        print( string.format( "Hello %s", person ) )
    end
end
local g = greeter("Jolie Rouge")
g()

How about this for weird?

(defvar *people* '(("Jolie" "Rouge") ("Angelina" "Jolie")))

That’s a bit obscure, notice the use of ‘, which is an alias of quote. Also, the variable name “people” has * on either side of it, that has to mean something right? Well, it doesn’t. The quote mark is a signal, and it means “That which follows is data, and not code.” The stars are just another kind of character that can appear in an identifier. That’s another issue that can lead to confusion with imperative programmers, because * is not an operator, it’s just a function, and it’s name is just another name.

Consider this equivalent statement in lua:

people = { {"Jolie", "Rouge"}, {"Angelina", "Jolie"} }

Notice how I did not use local? I didn’t because in Lisp, a variable starting and ending with * is conventionally meant to imply that it is a global variable.

I wonder how we might add to this list?

(push '("Jolie" "Blond") *people*)

Which has its parallel in Lua as:

table.insert(people,{"Jolie", "Blond"})

Already we are getting into the topic of structured data. A common idiom in imperative languages is the struct, which I will illustrate in C because Lua has no structs, per se only hashmaps.

struct Point {
  double x;
  double y;
};

int main(void) {
   struct Point p = { .x = 1.1, .y = 2.3 };
}

Which is a common enough idiom in C that it should be fairly obvious what is being done. In Lisp there is a little magic happening that can be a little confusing at first:

(defstruct Point x y)
(defvar p (make-point 😡 1.1 :y 2.3))

In this case, the function make-point is created for us when we define a struct.

So here’s a fun little snippet of lua that is functional but also object oriented (the function maintains state).

function CreateInstance()
    local self = {}
    return function(msg)
        if msg == "setMessage" then
            return function (s)
                self.message = s
            end
        elseif msg == "getMessage" then
            return self.message
        else
            error("Method missing")
        end
    end
end
local x = CreateInstance()
x("setMessage")("Hello World")
print(x("getMessage"))
x("setMessage")("Goobye Cruel World")
print(x("getMessage"))

Here is an alternative way, using the Lua “:” syntactic sugar:

local MyClass = {}
MyClass.__index = MyClass
function MyClass.new()
    local self = setmetatable({},MyClass)
    return self
end
function MyClass.setMessage(self,s)
    self.message = s
end
function MyClass.getMessage(self)
    return self.message
end
local y = MyClass.new()
y:setMessage("Hello World")
print(y:getMessage())
y:setMessage("Goodbye Cruel World")
print(y:getMessage())

In the first instance, everything is private. In the second, everything is public. In the second example, it’s possible to modify the code to have both public and private members, it’s also possible to create multiple inheritance. That’s already been well covered, here’s how we can do it functionally.

function TrapParents(real_args)
   local parents = {}
    for i=1,real_args.n do
        table.insert(parents,real_args[i])
    end
    return parents
end
function TraverseParents(parents,message,...)
    local res = nil
    for i=1,#parents do
        res = parents[i](message)
        if res then
            return res
        end
    end
    return nil
end
function MethodMissing(parents,message,...)
    return TraverseParents(parents,message)
end
function CreateParent1(...)
    local self = {}
    local parents = TrapParents({n=select('#', ...),...})
    return function(msg)
        if msg == "helloParent1" then
            return "hello from Parent1"
        elseif msg == "name" then
            return "Parent1"
        else
            return MethodMissing(parents,msg)
        end
    end
end
function CreateParent2(...)
    local self = {}
    local parents = TrapParents({n=select('#', ...),...})
    return function(msg)
        if msg == "helloParent2" then
            return "hello from parent2"
        elseif msg == "hijacked" then
            return "Muahaha, I has hijacked it!"
        elseif msg == "name" then
            return "Parent2"
        else
            return MethodMissing(parents,msg)
        end
    end
end
function CreateParent3(...)
    local self = {}
    local parents = TrapParents({n=select('#', ...),...})
    return function(msg)
        if msg == "helloParent3" then
            return "hello from parent3"
        elseif msg == "hijacked" then
            return "Not Hijacked"
        elseif msg == "name" then
            return "Parent3"
        else
            return MethodMissing(parents,msg)
        end
    end
end
function CreateInstance(...)
    local self = {}
    local parents = TrapParents({n=select('#', ...),...})
    return function(msg)
        if msg == "setMessage" then
            return function (s)
                self.message = s
            end
        elseif msg == "getMessage" then
            return self.message
        else
            return MethodMissing(parents,msg)
        end
    end
end
local x = CreateInstance(CreateParent2(CreateParent1()),CreateParent3())
x("setMessage")("Hello World")
print(x("getMessage"))
x("setMessage")("Goobye Cruel World")
print(x("getMessage"))
print(x("helloParent1"))
print(x("helloParent3"))
print(x("hijacked"))
print(x("notImplemented"))

We notice that the order of searching for a function is left to right. Since CreateParent2 comes first, it is searched first and therefore hijacks. We may want to do it backwards:

function TraverseParents(parents,message,...)
    local res = nil
    for i=#parents,1,-1 do
        res = parents[i](message)
        if res then
            return res
        end
    end
    return nil
end

Lua is by far the best scripting language I’ve ever had the pleasure of using. It’s joy to embed, it’s fun to use, and here’s the kicker: It’s more like Lisp/Scheme than you think. I recently saw an absolutely crazy implementation of linked lists, far crazier than the standard Lua implementation. Basically, implementing cons, car, and cdr gets us the linked list for free. The table based implementation may be faster. Note that this idea comes right out of the MIT Scheme lectures, and I believe it’s also in SICP.

function cons(a,b)
    return function (p)
        if p == 1 then
            return a
        end
        if p == 2 then
            return b
        end
    end
end
function car(l)
    return l(1)
end
function cdr(l)
    return l(2)
end
function cdar(l)
    return car(cdr(l))
end
function cddr(l)
    return cdr(cdr(l))
end
function cddar(l)
    return car(cddr(l))
end
local x = cons(1,cons(2,cons(3,nil)))
print(car(x))
print(cdar(x))
print(cddar(x))
local y = x
while y do
    print(car(y))
    y = cdr(y)
end

Consider the same definitions in Scheme:

(define (cons a b)
    (lambda (p) 
        (if (= p 1) a b)))
(define (car x)
    (x 1))
(define (cdr x)
    (x 2))
(define (cdar x)
  (car (cdr x)))
(define (cddr x)
  (cdr (cdr x)))
(define (cddar x)
  (car (cddr x)))
(define x (cons 1 (cons 2 (cons 3 '()))))
(car x)
(cdar x)
(cddar x)
(define (while x)
  (cond
    ((empty? x) (display "Done"))
    (else
     (display (format "~a\n" (car x)))
     (while (cdr x)))))
(while x)

In all fairness though, Lua would allow us to define even the loop the same, such as:

function whilex(x)
    if x then
        print(car(x))
        whilex(cdr(x))
    else
        print("Done")
    end
end
whilex(x)