I'm a massive fan of Clojure (I spoke at EuroClojure this year), and have used Unity3D for 5 years... I was also the developer of the first big iOS Haxe game, another unproven platform at the time, years ago.
I would be very concerned about straying too far from the garden path while also trying to ship a commercial game.
Unity's GC isn't even generational, a very bad mix with garbage heavy Clojure code. Expect confusing, bizarre and transient performance and memory use issues. Debugging will be a nightmare.
And on top of that, Stack Overflow doesn't have an unlimited supply of Clojure JVM answers; Clojure .Net through IL2CPP? Forget it; you'll have to figure everything out yourself.
It's generally a mistake to make the initial development more pleasant at the expense of long term maintainability; and I think that's the trade off on the table here.
I would concur and add that the main risk one runs in deploying any new, widespread abstraction to game code is an architectural meltdown, where perf bottlenecks are appearing everywhere, too many code paths have been locked in by the abstraction so you can't iterate on them fast anymore, you have race conditions you can't analyze properly, etc.
Iteration times across the whole development lifecycle are hard to optimize because in the near term the iteration problem is specific to writing basic functionality, then later on it's adding and adjusting a wide variety of assets, and then even later it's debugging and optimizing. Care too much about one and you mess up the others.
If you soak up pain up front to keep the core of the code in an unadorned imperative style, crunching on plain data, avoiding deeply nested callstacks or polymorphism, you get a huge assurance that you can handle anything that happens later on - it follows the grain of the tooling and the happy path of the optimizer.
The way in which I last approached the live-reload problem was to write a data model, and then bolt a scripting language on to generate data. Then reload the scripts - you get the fast iteration, without being particularly invasive on the main loop.
Edit: And similarly I've focused most of my abstraction braincycles on making it easier to access and swap out assets. One of the biggest recurring problems in engine code is deriving runtime assets from combinations of static assets and parameters. This has led me down the road of developing an addressing scheme.
I've just started to program a game engine and to my surprise the most difficult to structure aspect of the programming has been essentially, I'm not really sure how to word this, but mapping things to other things.
Making sure that a keypress eventually leads to the right function being called is a good example. It seemed really simple at first but then I figured that the keypress shouldn't directly call the function because then you'd need to recode or reparameterize some giant state machine because in certain contexts a keypress can do many things or control many different objects in the game (even controlling multiple objects at the same time, and what if you want to be able to control the game remotely or from two controllers at once?).
Before I started I thought the 3d movement and physics stuff would be the hardest... but I've found it was actually pretty simple, I just iterate through a list of objects and just call their respective member functions to make the movement happen and it seems to work fine for now.
This implements a state machine in Javascript. It's a very simple case with only 2 states, "playing" and "paused". Each state registers its own event handlers on entrance, and unregisters them on leave.
This approach avoids the need to answer the question "where am I? what should I do?" every time an event handler is called.
While I think the approach is nice, the code is not so nice from a superficial point of view. The more general problem is that event-style sucks. Look up CSP by Tony Hoare, and Occam the language. What we need is better formalisms for sequential programs that include non-determinism like user events (i.e. view the user as part of the program).
(And, of course, also parallel execution of these sequential threads, and also sub-states that can access parent states in a way. Look up Harel's Statecharts. A good example would be a form with two text input boxes. We could view them as parallel sub-threads of the form thread).
Nice, I implemented something very similar, except in my game not every controllable entity a) implements every control, and b) implements controls the same way, so I store what a control would do as a member function of each object and store an array with the names of these functions in each object. The indices of the array with the names of the functions match the indices of an array that stores the key codes I listen for, from there I match the indices, grab the name of the function and call it from the object being controlled.
In Javascript, you have to install event handlers. There is no control when they are executed, which is a problem.
When writing a game engine, you typically create events yourself at the top of the game loop (reading the input sources in a non-blocking fashion). You have better control what handler is called and when, but what handler should be called probably still shouldn't be decided in a central location. So I figure your object-based approach is a good solution. (The individual handlers can rely on the fact that they are only called at the start of the loop).
I realize a Javascript app could easily emulate the game loop style by letting handlers write to a queue.
Yeah. There is a lot of routing/binding action going on in an engine when you get past the "hardcode everything for a single context" stage. I'm currently mixing about four big ideas together in my engine - they haven't all been deployed in the actual game project I have right now, but they're getting developed alongside what's there:
* Unix-esque hierarchical path routing - the addressing system I alluded to in parent comment. It's used as a mapping substrate - you can swap out parts of the route to quickly change what's being addressed, and it stores just a little bit of data so that leaf nodes mostly act as keys into other containers. I changed it from int-per-node to int-array-per-node today, which may or may not have been a great idea, but came up because it was irritating to have to make a special store for string keys.
* Micro-database system for allocating various types of engine data. This is the biggest source of friction in the engine right now - each data type has a custom container, and in theory, when allocated it'll automatically map to a path address. Engine scenes are currently loaded up through a script saying "allocate these things, relate these things together", but the addressing isn't present, you just access the containers more-or-less directly(ad-hoc per container).
* J. Paul Morrison-style "classic FBP" to compile data graphs. This is more of a "I want to program with Lego Bricks" kind of goal - it's the most directly interesting one for the keybinding problem, and it's used in existing engines for aspects of rendering graphics and sound. But it ultimately depends on having the addressing system functioning on real assets, so it's on the backburner.
* A "stack of small compilers" as in the VPRI STEPS project. This is used to turn game design assets(event scripts etc.) into runtime data by way of a behavior tree compiler that emits opcodes for a small VM interpreter. This is the stuff I'm really using the most, right now, and it actually inspired the addressing system, as compiling all that stuff created a need for it.
Have you seen the Game Programming Patterns book (http://gameprogrammingpatterns.com/). It was originally a blog that later become a book, and I think you can browse most of the content online. It's full of interesting lessons and patterns and deals with coupling and state management. It won't have the answers to copy and paste, but it gives you a lot of food for thought and it's really well written.
I'm not a game programmer by trade, but I'm interested in that stuff and I really enjoyed it all the same. It made me rethink some stuff about enterprise patterns too.
A keypress should never do much other than to update some game state to say that whatever key was pressed and it is mapped to whatever game concept in your input mapping. Nearly any code I've ever seen with an if/keypress, do this is an anti-pattern. Once you have your state, you just need to handle it somewhere later during a game tick, and keep in mind that might be even multiple places.
I would recommend thinking in loops, so it sounds you're already on the right track, you just need to expand that more to everything else. You generally want to do the same "thing" to many "things." Since you are newer to programming engines, I won't get too technical and I'll say that the gist is that the CPU likes this. The bi-product is if you take this approach, you'll tend to have better architecture as well as performance and solve some of the problems you listed. This is really the basis of an entity system approach if you want an example architecture.
Whether using entity systems or something else, my recommendation would be to separate out functionality and make them rather agnostic to each other. Again, this fits in with the loops paradigm. You still often need things to communicate as your problem suggests, but the way to do that is to often take some state, update it with new state, and pass it down the chain to see if anyone cares. That chain itself can be a loop, i.e. a bunch of systems in your game loop calling a method like update(tick, state, ... ). That said, you also need to be careful what and how much state gets passed around for performance reasons (ex: avoid excessive copying) and to prevent weird things from happening if you start introducing concurrent and/or parallel programming. To that end, I tend to keep things minimal as I can, or at least have some sort of way to hand things just what they need, not everything.
Queues also relate to all of this and can be your friend. A common communication approach is to use mailboxes for example and tends to be more straight forward to debug and optimize than using callbacks and event-based approaches (they tend to murder the cache and your stack traces). Queues also play nicer for being building blocks for concurrent programming.
Also keep in mind you may need to flip your thinking. Sometimes you don't actually need some action that happens during a tick to be processed the same tick. You can often or will often want to defer things to the next tick, or even several ticks after that. There's a Naughty Dog presentation on Last of Us port to PS4 that relates to this and how they flipped even the way frames are processed on an existing codebase to get better performance. The point is not for you to do what they did, rather to start thinking in terms of what the user will see as the end result. The user often doesn't care that you didn't finish processing some action last tick if it had no bearing on the gameplay.
Thanks, right now every game tick I check a sort of register to see if any keys have been pressed, held, or released and then immediately call the associated functions to implement the controls. If I'm interpreting what you said correctly and I understood the Naughty Dog presentation correctly (I did see that one!) I guess I should instead add those function calls to a queue and then call them at my convenience?
That makes sense to me, I could split up the queue and send it to different threads to be worked on or perhaps delay doing things in the queue if a frame was taking too long.
I wrote a long comment, but was rate limited at the time.
I'll summarize instead with a simple list:
- It's better to map your input to some game actions than pass around the raw value
- You don't need to do anything creative like queue function pointers themselves, just the raw values.
- You can have an input "queue" but that is typically different from a queue that is going to act on those inputs.
- You need to deal with multiple inputs potentially, like someone pounding a button. Hopefully your input lib deals with this some already, but always be aware of validating the input and deciding which one matters most. To that end, priority queues are good sometimes. You also sometimes want to cancel inputs. If you want to read more about it all, I'd read some about "intention systems" as related to input.
- Once your input is mapped, you can operate it in multiple pieces of codes, systems, etc. that might be downstream in your game loop
- You can transform an action into yet other new state as you go along
- Queues are indeed great for doing multi-threading, just be sure you select the appropriate kinds of queues. Mostly this ends up being thread-safe queues that are lockless. But sometimes you want to just use queues for FIFO, and these queues are generally faster and/or more flexible if they use a non-thread safe approach.
- You can have many queues in your code as mailboxes to distinct systems. Again, think something that is agnostic to the world around it and just receives some state it needs, and has its own "inbox" of things it might need to process this frame. It certainly could be that you can't process them all that frame.
- Keep in mind with queues generally once you pop something off, that's it. So you need other ways of hanging on to state and things that aren't ready. To that end, simple arrays are your friend, especially if you can pull things out quickly by index in relation to some id (ex: entity id) and keep them tightly packed. Dynamic vectors are garbage and mostly not used in serious game engines unless the developer had a wtf moment or the use case specifically calls for it. That's a longer topic though.
- Attaching functions directly to input again is a pretty bad approach. If you need more than one function, you'll have to implement something more complex. If order of function calls matter, you again need yet more. The order should be well-defined in the game loop itself.
- Regarding order, it also relates to communication as well. Note that you don't need to always process every new input or piece of state every tick. Sometimes you actually don't want to do this and defer it to a later time. There's no 100% rule, but an easy thing that helps is to think if the player or game sim will suffer because you didn't process that state during that tick. For some physics related things, that can be bad, but for other things, it's more of a "as long as this happens very soon, it's alright."
Authors please heed the above warning. You are entering an extremely competitive, challenging marketplace and opting to play on "hard mode" for limited benefit. Worse you're doing it with your first project. You should focus on becoming an expert with vanilla Unity and shipping as many titles as quickly as possible.
So I've shipped a few small games with clojure and unity. GC does come up, but it's very easy to optimize your tight update functions with Unity's profiler.
As for finding answers, the clojure CLR interop is pretty similar to JVM. I spend most my time looking at the Unity scripting docs.
I couldn't agree more. If anything, the pleasant experience argument tends to come up more in the context of discussing a secondary scripting language to be added on top of a fast, already working foundation. Even then, you mostly just end up using something reasonably fast like Lua or one of its cousins. I can't see Clojure really being a good fit for any existing engine I am familiar with as a main or scripting language (maybe ClojureScript if we stretch it for something using JS, which I disagree with), and I love Clojure.
FWIW, scanning the Arcadia source there's also some garbage being created that can be easily avoided. I didn't really trace to see what each thing was "doing" or if it even runs during a game, but it certainly felt eye opening to me to see lots of temporary garbage created in something that is supposed to be lower-level. I am not sure at this point what would get optimized away anyway (ex: foreach loops, extra tolist/arrays used to create garbage in the past), so here's to hoping the devs just didn't get around to optimizing that stuff away yet. Seems to me you would want to minimize garbage from Aracadia itself given Clojure is going to create yet more, not to mention cut stylistic corners for pure speed in this context.
To your first point, Mooi is planning on using Clojure as a scripting language. Clojure compiles to CLR bytecode, and is in theory as fast as C# is as a language. Using the idiomatic persistent data structures has an overhead, of course, but Clojure allows you to be flexible with how and when you do that. I wouldn't say Lua or JavaScript are "reasonably faster" than Clojure is on the CLR.
To your second point, we're aware that there's room to optimize and we're actively working on it. We profile and optimize our code paths regularly, and the results are often surprising. If there's "obvious" stuff left in there it's because we're tackling bigger fish than e.g. a foreach loop. None of these are show stoppers, in practice, but this is part of why we're in pre-alpha.
I am aware of the compilation into CLR bytecode and I've been a .NET user since the beginning, though I don't work with it much these days. The fact that it someone is using Clojure though doesn't mean it will translate always into the most efficient byte code. On the JVM, this is a problem sometimes forcing you to either write your Clojure differently to produce something more efficient, or requiring you to write a bit in Java and call it from Clojure (contrary to amateur practices w/Unity, people do need to share code that is not just abstracted by the engine such as some behavior). The same paradigm would be true of having to drop to C# and then call it from Clojure CLR. The problem here just becomes you get further from what is good about Clojure and instead write a version of Clojure that is closer to C# or something else.
Regarding speed, I think you misunderstood part of my comment. I simply mean you pick something decently fast when picking a scripting language with regard to the host language. Obviously Clojure is decently fast on the CLR in theory, but in practice may be another matter given the properties of a game which are far different than general use computing.
As for the speed with regard to Clojure, I've covered much of the general stuff in other comments, but I was in part writing to the point of using it independent of Arcadia in a scripting context. For this, I am contending there aren't many advantages to using Clojure over other approaches as a general scripting language for most engines, ex: Unreal, CryEngine, in-house, etc. Unity obviously benefits because of the CLR, but there are still issues with both Clojure CLR and using something that is yet another abstraction on top of Unity as others have pointed out. This makes life hard for newer or less-skilled game devs especially which it tends to sound like the original post authors are in general. More than anything, unless you have unlimited time, do you want to get a game done or do you want to help fix a new runtime?
Additionally, I am not comparing Lua to Clojure for the CLR specifically, rather stating that most of the time you add a scripting language to an engine in general it depends to be languages with certain properties. The reasons have little to do with Clojure as a language and more to do with scripting specific needs, which Unity itself doesn't really handle very well depending on how you define "scripting." That is any decent scripting engine must address some of the following below in a game, or it is not a scripting engine but rather just part of the main compilation target.
- Support hot code reloading, ex: from file system, user scripts, in game commands, etc.
- Compile/JIT reasonably fast if required
- Provide bi-directional calling semantics
- Easily integrate with the primary host language of the engine
- Decent experience for script authors
- Easy tool integrations, including with 3rd party middleware
- Minimize garbage if generating any, and make collection predictable if possible
- Not require the game code core to be recompiled in tandem (a primary motivator for using one from C/C++)
Clojure on the CLR certainly does some, but I'd argue not all well to be a huge benefit over C# in terms of productivity, performance, user experience, or user-facing scripting features. Of course there are some cool things due to Clojure's reader and macros, REPL, and so on, but the point is more the aggregate benefits. The reason people use Lua, Squirrel, AngelScript, and many others is not that they are the best, or fastest languages, but are generally good in all those points with regard to the primary language of the engine they are attached. Personally, I'd rather use other languages like Clojure, but there are tons of drawbacks that don't bite you until you are working on a real project, many that I listed elsewhere in this thread.
As I noted (and you repeated), I assume the code is alpha and there are still items to address. Nonetheless, most of these things are obvious and easy up-front, and many well-known in the Unity community, so I'm not 100% buying everything is a "later," just some of it. While in a normal game I wouldn't expect various optimizations or micro-optimizations, in an engine you definitely do. Even when working in C or C++, all the engines I've worked on we used our own allocators, minimized general allocations, added indexing tricks, branch prediction and inlining tricks, array packing, alignment tricks, and so on from the start, because that's the job of the engine and the end-user (game dev) cannot optimize away all of those things if the engine is the issue. Of course Unity doesn't do all these things well and things like Clojure layers are a lot further from the actual engine, but I'd argue that it means that everything needs to strive for equal parity or better performance when and if possible. Those are things you design around mostly from the start, the other optimizations that I think you are referring to happen later. I'm simply contending I see a mix of both.
Finally, following up again on the use of Clojure, as I've said in this thread, I'm a big proponent of Clojure. That aside, the CLR vs JVM implementations do not have parity because of lack of resources more than anything else. This automatically puts a lot more work in the pipeline and ultimately on the game developer, at least until things mature more. This isn't about using something like Arcadia at all, but rather using it right now given the original constraints and resources the article author mentions or implies.
F# with monogame will get you better performance and less weirdness, it is a pleasant lighter weight environment. unity gc and runtime are rapidly improving now that ms owns xamarin though.
they have a roadmap with the plans laid out on their website. it was impossible in the past because xamarin did not want to sell them licenses for newer mono. now Microsoft owns xamarin, and they are happy to. some improvements have already landed at least in beta.
Unfortunately, although it's on a page named "roadmap", it doesn't look like a GC upgrade is part of any concrete roadmap. It's in a big bucket labeled "research".
On topic, I just want to affirm that it's hard to overstate the inadequacy of Unity's GC, or the amount of time one can sink into optimizing for it.
The GC upgrade is going to happen when they upgrade Mono, and the Mono upgrade is definitely going to happen, although I don't know if it's going to be before Unity 6. (Source: Unity people have suggested this to me.)
Does anybody here have experience with F# and Unity? Unity uses older versions of .net, and F# is not officially supported, so it's hard to find real experiences with it, although it sounds very tempting. I would imagine that it would be most effective to have the performance-critical stuff in C# and then F# for developing higher-level game mechanics.
> I would be very concerned about straying too far from the garden path while also trying to ship a commercial game.
Agreed. OP is choosing the dangerous path of picking heart over reason.
It's not just that using a dynamically typed language for a game is suicidal, it's that you don't even have a choice in picking a statically typed language: C++ is pretty much the only credible game in town if you want 60 fps and a chance at not being ridiculed technically before game testers even look at your game from a purely gaming standpoint.
In this particular case, the game is being run using modifications to Unity, a very well-established game-engine, so C++ is already doing the heavy lifting (alongside Unity's way of writing GPU shaders).
Clojure is being used in place of the usual C# for game logic scripting and the like. It's much less performance-critical and so using something like Clojure is a lot less nonsensical.
Whether they're optimizing for the right things in picking Clojure over some other language is maybe up for debate (I'd guess their way would be easier to iterate changes, but harder to debug, and the clojure-on-unity stack they're using is pretty much alpha quality), but Naughty Dog already showed that commercial games can use a Lisp in a similar part of the stack with some measure of success.
>> It's generally a mistake to make the initial development more pleasant at the expense of long term maintainability;
Me and my friend are working on this game [1]. I have prototyped all the game logic in Common Lisp (using both LW Mobile trial edition and MOCL). We could really quickly iterate the game play and reach a design where the experience will be fun.
But to get it working with 60 FPS, we are writing the game now in Swift. A lot of our game asset pipeline is also in CL and Python. So as per my limited experience, I would prototype the game idea in CL; also use more powerful tools for offline utilities etc but ship the game in whatever is the best toolkit for that platform. In our case, Swift, SceneKit and SpriteKit.
I'm one of the programmers at Mooi Studios. We intentionally avoided being very specific in the article because we were expecting some skepticism :) Because of this it might sound like we made this decision without putting any thought into it, and that's not the case at all.
I don't want to argue about what is the best tool for the job, because honestly I don't think anyone can say something like that and be 100% certain about it. All I can say is that I've tried those tools before, and I'm pretty sure I don't want to write software that way anymore.
We are aware of the challenges, and we know that using C# would be easier in some ways. But enjoying what we do is a big deal for us, and we're willing to do some extra work to get there.
Ultimately, whether our games succeed or not has little to do with the technologies we use. I just finished Hyper Light Drifter which is probably one of my favourite games of all time, and they made that with Game Maker.
Anyways, for now we're only using Clojure to replace C# for game scripting. We're not writing a full game engine or anything with it (yet, hehe). We believe the differences in performance for these purposes should be neglible, and initial testing seems to confirm that. The gains in simplicity and workflow are real and are huge. And most importantly we're enjoying every minute of it.
I've been hacking away in as my job Unity for years..
Can someone tell me do they think the practice described in the article is a good idea? (The writer says they are just learning so their opinion is not very valuable to me (When theyve cranked out a bunch of games on various platforms and encountered all kinds of horrible shit and overcome it, it would be more valuable to me))
I'm all too familiar with OO's many foibles and I'm interested in functional bur have never done any.
One thing I've heard though, is that OO is bad for most stuff except games. Where you actually have this world with all these entities composed of various objects going around in it about their business.
Its does make sense to have a 'Cow' object that has state + methods to get it to do various things like 'moo'
Additional: Looked around a bit and the Arcadia repo says..
"
Arcadia is alpha-quality software, and shouldn't be used for anything important yet.
"
I'm also try as hard a possible and use as much standard, vanilla Unity stuff as possible, as the more weird custom plugins and whatnot you use the more heartache you get when it comes to building and releasing + updates on a million damn phones or PC setups. Anything not standard has to justify its existence against very jarsh criteria.
I will download and mess with this though. I'm actually really excited to do some functional programming!!
I've programmed mostly-functional stuff for a while in a bunch of languages and I have been working on a large performance-intensive (massively multiplayer, VR, and mobile!) Unity codebase for the past year.
My conclusion is that functional programming is still a good idea to the extent that you can do it. Specifically, you should still minimize and encapsulate mutable state, and you should write pure functions when possible. However, the infrastructure doesn't exist to do functional programming efficiently in a lot of ways. Examples:
- Most of the .NET standard library functions over sequences (i.e. LINQ) incur substantial time overhead and do a lot of heap allocation. Writing and using generators, same story.
- Anonymous functions incur heap allocations and overhead which does not seem to ever be optimized out in Unity's Mono runtime.
- .NET's standard library data structures are mutable.
- Most of Unity's API and the libraries people write for it are not functional.
I don't think you would be wise to try to fight these facts and paper over everything with functional abstractions, or you will die of a thousand small performance cuts.
Just a note, this is mainly an Unity problem as they use an old Mono compiler and runtime. Additionally lambdas only create garbage when you capture local variables, and the normal GC is optimised to run without you noticing collections.
The latest C# also has a whole library dedicated to functional data structures [1]. Additionally the C# compiler was rewritten in a functional style (proving it's capability IMO) and is completely immutable. It made sense for their use case as they can pass the syntax tree around to other threads now without fear of it changing.
John Carmack has discussed functional programming[1] and has spoken about using Scheme at various points. So, the basic idea of using functional programming for games appeals to at least one well known developer :)
Arcadia is not representative of all functional programming practices ever. Its alpha state isn't reflective of the alpha state of functional programming either, which has been around for longer than most of us (50s-60s).
FP boils down to one thing: referential transparency, or purity. A pure function f is such that any call to f(x) always yields the same result, to the point where any call to f(x) can be replaced with that result with no change in behavior. With it, we also typically use first class functions, higher order functions, closures, and lots of recursion; those aren't the key purpose of it though, I'd argue.
Even in OOP, we're taught not to have globals. Why? They're not thread-safe; it's not clear who owns them; anyone can change them at any time; in languages like C++, there are also lifetime concerns regarding ordering of initialization between translation units. Referential transparency, I'd argue, is pushing the same exact agenda: stop modifying global state. It's not safe and it's not pure. As John Carmack mentioned in the article above, as game developers, we keep so much state in our mind at any given time; this grows exponentially with the number of threads we're also thinking about.
Even in the most OO languages, we can strive to write code based on pure functions to make those functions easier to reason about, test, and compose. When something goes wrong in your function, and it's pure, you need only worry about your inputs, what you're doing in your function, and your outputs. When it's called doesn't matter (such as before or after some global state has changed).
Rich Hickey, the creator of Clojure, said, in his Spec-ulation rant, that it's tough to explain to Java developers why they should consider Clojure. The reason it's tough is that you first need to explain to them that there's a problem they have which they don't realize: an anxious burning sensation regarding all of the mutation going on in their complex systems. When people use Clojure, and similar functional languages, they often feel liberated that an entire class of problems and bugs can be ruled out. With pure functions, systems can be much easier to understand through a narrow lense. With immutable data structures, tasks can trivially be run in parallel without the worry of thread safety; values don't change. As someone who came into Clojure from a background in C++, I've found that it is, indeed, liberating.
> One thing I've heard though, is that OO is bad for most stuff except games
OO is useful when your primary abstraction is modelling the outside world. This is often true for games but is also true in a lot of other contexts. The explosion of interest in OO languages coincided with the explosion of interest in GUIs as OO was found to be helpful in modelling documents, windows etc.
One of the reasons why C is still heavily used, despite its well publicised faults, is that it's proved useful where your primary abstraction is the machine you're running on.
Functional programming, in my view, is useful where you want to abstract over operation. Where you want to specify what you want to do rather than how i.e. declarative programming. That's why concepts like map, filter etc have had a fair amount of "crossover" success.
At different times, all these abstractions can be useful. With experience we have learnt that some features associated with each type of programming are more valuable than others e.g. multiple inheritance and raw pointers are used with caution.
For something like a game, assuming it's non-trivial, I would expect that you'd find yourself needing to draw from all these. So I would be cautious about picking a language that cut you off from one or the other.
a long time ago i used to write code in something called 'AVS' for Winamp... it also let you modify code at run-time, but it was certainly not functional.
you can actually do the same things with C#... and unity does let you do it out of the box to some degree. there is no reason why that couldn't be further developed though.
You can argue that lots of games can/do modify things while running. Entity systems allow components to be added at runtime. This not only allows for different code to be executed, but also I've seen code that reads components and hot loads additional code for systems supporting those components if necessary, which can be done a number of ways, and raw from the source even.
Smalltalk also supported this and isn't really functional. In addition to the Smalltalk environment itself pretty much being a live environment, I saw things like serializable continuations in Smalltalk. You could do fun things like a Gemstone Smalltalk app I saw that would take the image state and allow hot modifications via a bug tracker.
There's tons of other systems for doing this. And it's also a reason why many games write big chunks of the type code you would want to hot load in scripting languages like Lua.
There are tons more examples and has no relationship to functional programming other than to say that it was a common thing in the Lisp world many moons before most of these other things existed.
Whether this they stick with Clojure/FP or not, isn't it important for them to at least try. Sure maybe FP isn't the best tool for this job right now but maybe someday it will be. However as a development community if we don't even try to use it for real projects it will never get better.
Perhaps through their trials and tribulations they will gain the knowledge to aid in the development of Arcadia/ Clojure + Unity. I just feel like there is a lot of well meaning but discouraging comments here but I think people often forget the science part of computer science. Sometimes you just have to experiment and fail and experiment some more to push a paradigm farther.
I don't know how successful they will be with this approach and maybe they will end up ditching it but even if they fail the first time they have the opportunity to gain valuable experience.
Exactly! I'm one of the core devs, and we all see Arcadia as an experiment in what game development could be. Some people want to expend effort mastering the tools of the day, and that's totally cool, but we're somewhat more motivated to spend our time trying new things. Members of the community have already pushed out a number of games, and their experiences have guided made the tool better and better.
As a semi-retired professional gamedev, I don't get this. Let me qualify what follows by saying that I currently program quite a bit in Clojure and ClojureScript on my personal/side project, and I love both languages. Additionally, Lisp was one of my first languages several decades ago. Sorry for the length.
Game programming like most programming largely requires choosing the right tools for the job. While I agree it is important to factor in the skills that you team has, the choice of Clojure + Unity + this write-up seems like retro-fitting for the wrong reasons. Regarding Clojure, I can appreciate what Arcadia is trying to do, but it seems like an odd choice for an unskilled game dev for anything but messing around/intellectual fun at this point. This is both because the software is quite honest and labels itself alpha, and because Clojure and Unity is not necessarily the best combination IMO for many reasons. The primary reason I'd contend this is because it is a huge pile of abstractions of abstractions with not enough payoff. Even just browsing Arcadia's source, I can also already see a few bits of code that aren't optimized as well as they could/should be for something like this. I can only hope the article author is not serious about pursuing a commercial quality game of significant complexity and power with this approach (i.e. not necessarily AAA level, but upper-tier indie).
As for functional programming in games, it is a big subject and an interesting one, not to mention a worthy pursuit. Indeed, huge amounts of the trends in game programming for the past two decades at least have been moving to functional paradigms. Entity systems are a great example of this. One way to think about Entity Systems in particular in relation to a functional language like Clojure is that it essentially allows you to think about your game state as a reduction of states (entities with attached components and systems operating on them) over time. Furthermore, as multi-core programming becomes more important, some of the constructs that make concurrent programming, or even parallel programming easier are commonly found in highly functional languages. Clojure is no exception and has many things are amazing to this end, but not all are necessarily useful in the context of a non-trivial game. There are many more things I could touch on, but suffice to say that programmers are bending over backwards to make C and C++ in particular, followed by C#, Java, Swift, and other languages behave like functional language. Ironically, I think part of the problem with most functional languages is that people are pulled from the other end and forced to make these languages behave like C++ to get acceptable performance and other traits.
Despite many OO/imperative languages used in game dev moving towards more functional trends, the promises, power, and tools that most of these languages provide are vital. There are too many reasons to list (some good, some bad) why C and C++ are still dominant in this world for serious game dev work, but a few include existing tooling, full control of memory, ability to work around language induced performance roadblocks, possibility of portability, optimization per platform if required, bountiful workforce, and low-level integration points with hardware, SDKs, and tools. That's not to say other languages don't have these things, but rather they have some critical flaws in some of these areas that make life harder than it needs to be at the cost of actually getting a game completed, which in the end is one of the most important things.
I know it's frowned upon, but I'm splitting this rather large comment into two parts.
Clojure suffers some because of the reality of today's game dev ecosystem. Just looking at the source of some past attempts to wrap things like LWJGL or OpenGL (hint, none of these tend to look like Clojure anymore) should be enough to answer some of the questions why this is. There are many others reasons as well that only come to bite some people until they are in the middle of a project. Beyond those reasons, even just dealing the JVM or the CLR for a game present significant challenges to a developer.
Unity itself has had to work around challenges too, and the abstractions and development comforts it provides are not free. Even experienced Unity developers who use C# still have to be mindful of how they write their C# (i.e. don't write the prettiest code even if you can because it will punish you). Of course this is true of any language and for other platforms like in mobile game dev. The point is that it's especially limiting to have additional layers of abstraction largely beyond your control. Eventually, it reaches a critical mass where you're writing this weird meta-language to do what you want because writing a game with the ecosystem around you forces that instead of letting you write idiomatic code in the language you picked.
The further you get away from the metal for a game, no matter how simple, the more you will face problems. It's nice to use languages like Clojure, Python, Ruby, JavaScript and so on for games, but for serious work they often get in your way. For instance a common problem the average developer encounters is the game loop vs. frame rate - how do I get enough done during a tick to not grind the game to halt? Garbage collection, de/allocations, and so much more become your enemy and you start to feel like you're fighting some kind of magical force trying to slow down your game or make it less predictable, rather than being productive or even optimizing it in sane ways. And yes, predictability is vital to writing a good game, because the last thing a player wants is your game to do stupid things at inopportune moments like in the middle of a jump, never mind other concerns like debugging, multi-player, or platform requirements.
Of course there are workarounds for many problems you may face, but as game complexity grows, things tend to scale out of control for most people. Many of these problems cut so much in to the time or make you have other sacrifices that you start to feel like you're largely missing the benefits of working in these alternative abstractions. At some point you just end up breaking all the rules of your language/tools/libraries to get the game to the level you want. Worse, you're working on many problems that are quite far from actually finishing your game. Obviously for simple projects, much of what I've mentioned previously is not a problem, just to again make that clear.
Getting back to Clojure, I feel it really suffers from the aforementioned issues for non-toy games. This isn't an indictment of Clojure, just about picking the right tools. Immutability, atoms, refs, agents, CSP, sequences, transducers, recursion, and so much more seem like they would allow making a game quicker, easier, and with less headaches. What ends up happening to most people I've seen who try to use these kinds of tools, whether it is Clojure, Lisp, Haskell, Elixir, or anything else is that at a certain threshold of requirements, is what I mentioned earlier - it all falls apart. At this point, you spend all your time removing all the goodness the language and tools provide. You start writing your own libraries, often down to numbers, matrices, etc. because you have no other choice if you want things to run in a sane, predictable way and to integrate with anything like OpenGL, input libraries, hardware, SDL, and so on. You throw out immutability in huge parts of your game, and you realize that refs, agents, channels, sequences, and more are just making life worse, not better. Pretty soon the entire language is stripped down into something almost unrecognizable, left with only a few core nice things. You then descend into the next layer of hell and start porting things into Java and calling them from there. Even in Java this can happen to a large degree. Add in more unpredictable stuff and abstractions like Unity, Unity plug-ins/add-ons, multi-platform requirements, talking to other libraries you want to use, and so on. The author of the article mentioned simplicity as a selling point, but for non-trivial contexts, you will almost certainly throw simplicity out the door. It starts small and snowballs as I described.
For someone building a text adventure or other simple game, you probably don't even need Unity anyway. If you're building a smaller indie game or want to get something done quickly, just use Unity and C#, and you'll get it done quicker and benefit from the ecosystem better. If you can't/won't learn C#, you shouldn't be programming or making games. I know that sounds cruel, but at some point we all need to acknowledge our skills. A game developer should be able to learn any language and be productive in it quickly. The average game dev may not touch the entire game, but more sophisticated games often use several languages, especially if you count things like shaders and scripting engines as being distinct.
If you're just learning/new to game dev and/or really want to learn Unity, just use it as intended, otherwise you're adding more layers of abstraction and complications that make it actually harder to learn anything, and worse to get things done. It may often seem like you figured something out and using your favorite tool will get things done quicker, but most of the time you'll hit the ugly thresholds I described when you try to combine it with something more sophisticated like Unity. Use things as they are intended. If you want to make a game in Clojure, great, just keep it simple, write your own minimal engine optimized for Clojure or hope someone makes one someday, or go the ClojureScript route to again make something simple.
In summary, Clojure is indeed an awesome language and you can write a game with it, just I wonder what is to gain using it for Unity. In the general sense of things, I wouldn't recommend layering too many abstractions when building games. If you feel otherwise, I'll refer you to the graveyard of projects that have tried to take X and make it work with Y - it is a huge, sad place. That said, I'd love to write a non-trivial game in Clojure or another functional language one day, somehow.
I haven't done much functional programming, so I have a question regarding how this is done.
From what I understand, functional programming usually avoids side effects, so mutable data structures are replaced by immutable data structures. So instead of modifying an existing data structure, a new data structure is created containing the modification.
So how does this work with games? When you modify the game state, is the entire thing copied into a new game state? Doesn't that really slow the game down when the game state is large and changes are frequent? Is there some trick to this that avoids unnecessary copying?
Some immutable data structures implementations perform optimizations under the hood.
For example, if you have a large game state object with mostly stable fields, but the position value changes frequently, you don't need to perform a full copy of the object when position changes. Instead, you could create a new object that only keeps track of the position field, and delegates all requests for other fields to a more stable underlying game state object.
You could also imagine an environment that, if you're going to throw away the reference to the old game state anyway, then it's safe to mutate the object directly instead of bothering with copies or proxies.
…that said, most implementations—especially if you're not careful—will probably incur performance costs, and, at 60 FPS, those matter. They're just not necessarily as bad as you're imagining :)
Nobody has ever answered the question of the competitive advantage functional programming gives apart from the opinion that "sounds cool". The principles of functional programming definitely have their place in modern development but picking the de-facto tools for a job is always the safest choice when you do commercial applications.
It's kinda of sad when people treat functional programming not as paradigm but as a solution to their problems. Sometimes I feel that they think they will let all of the bugs, design decisions etc just go away, only by using FP.
I think that for game dev OOP is a better abstraction than FP.
One of the great things about FP and Clojure in particular was that it showed me how you can cleanly separates values, identity and memory addresses.
But in games you're often dealing with memory directly anyway and don't want to abstract that away. And abstracting identity from address doesn't buy you much because you're always running in shared memory space unlike distributed systems where it suddenly makes a lot of stuff transparent.
First and foremost, simplicity. The code is shorter and more manageable.
I don't know if functional program is SIMPLE as much as it is CONCISE.
I didn't see the word monad, fold, or reduce anywhere in the article. Is a functional programming language used with side effects and all? There's no acknowledgment of scenarios where managing the state the functional way can be unnatural, and require an intellectual hop. Or, alternatively, that this game in particular lends itself more naturally to the functional paradigm.
Clojure generally doesn't use monads and has its own way of dealing with mutable state. Clojure doesn't prevent side-effects, but it does make them somewhat unidiomatic in most cases. In any case, Clojure makes it pretty easy to write pure-functional code and carefully manage the mutable state and side-effects that you have.
Having said that, I don't know anything about Arcadia and how the Clojure code interacts with Unity and how clean the paradigm separation is...
Modern C# only has to be written object orientated if you want it to be, and works pretty well as a functional language if that is what you prefer.
Biggest problem with Unity and C# is the delay in adopting the latest language version - last time I checked Unity was C# 5, not 6. Hopefully Unity adopts .NET Core soon, getting around this problem.
Did they say they were going to .NET Core? As someone who researched porting code from .NET to .NET Core, it seemed like a lot of work, there was a lot of libraries we relied on that just didn't exist in Core, and we decided not to pursue it within the company at this point. I imagine such a massive engine as Unity would be even more difficult.
I know your pain, and there is also a big shift regarding dev tool chain as well. But once all thats over, I've found the resulting framework and experience pretty superior.
Especially when working across desktop/mobile/web/docker domains - using core everywhere works pretty well.
Just a nitpick: you can live code in imperative languages and functional languages can require compilation. That has more to do with your design, framework, and tooling than the language itself. It mostly is a divide between compiled and interpreted languages, though that distinction gets blurrier over time.
That said, Im a big fan of functional languages for game scripts (and got in to that myself using LISP on text MUDs).
I was only a lowly content builder and quest manager, so I can't speak much to how the server mechanics worked.
But essentially, each entity (usually a mob) would have a set of event scripts (attached to some kind of internal lookup table), and each time an event happened, it would call the appropriate scripts/function/macro.
So what I did was logic such as "game tick -> wander one step in a random direction" or "player enters room -> attack", but also slightly more complex ones such as "combat round -> if low health, flee".
The cool feature of the server was that you could specify multiple options in the template, and it would randomly pick one for a newly created instance, giving some variety to behaviors (eg, some guards flee and some fight to the death). Each instance could also be given a specific one after instantiated, which was how we constructed special events. (That is, use an existing template, tweak a couple behaviors of the instance, and then redecorate it.) This is mostly just a benefit of first-class functions and LISP's ability to switch strings to code.
LISP was mostly just nice in that you could often one-line fairly complex behavior, so you could build in-game over telnet.
Honestly, I really miss text MUDs, but does anyone actually want to read anymore?
I too was a mobprogs builder, but I hung around diku-based muds, and none of those used lisp. I loved the SMAUG codebase in particular. There was a NO_FLOOR property, and I discovered that there was an brief input delay on hitting the floor, during which your commands would be completely thrown out of the buffer. If you made an On Entry script in the destination room to immediately teleport the player back to the room with no floor, you'd trap them in an infinite loop that didn't buffer overflow, and completely ignored their commands. They would fall, hit the floor, and before their input block had ended, they would be falling again.
Now, did you know that SMAUG's mudprogs ignored level restrictions, but could be activated by any IMM's force command? So when you force the mob to mpteleport a player, it doesn't check that you're only level 55 and the mpteleport target is level 65... so I called that a prison for gods.
I think I spent more time trying to break the code than actually producing content.
For anyone reading: this is basically how all hacking works and why security is hard -- you have to think out every way several components might interact.
I love text games. Looking forward to taustation which is going into alpha soon and written by a prolific perl author. Good stuff. Check out the blog articles.
This may not be what you're looking for, but it's a nifty example of building a small text adventure system in Racket and then leveraging its language building tools.
Fair, but there's an important difference between a tool where live coding is technically possible and a tool designed to support it. Clojure is built from the ground up for the REPL experience and supports e.g. redefining functions seamlessly. The fact that it's functional helps a lot, too, since that means that the order things are defined or executed in matters less.
Python is an example of an OO + imperative language with a built in REPL.
My point was that it has to do with if there's an interpreter for the language, and if the interpreter works with your toolchain.
And that with the advent of JIT, that distinction becomes increasingly blurry, because the techniques to compile and link small blocks of code can be used by languages without an interpreter, per se.
I believe that on Unreal Engine 4, you use C++ and it does have a feature that mimics "live coding" where it recompiles and hot-reload your game even while playing.
I would be very concerned about straying too far from the garden path while also trying to ship a commercial game.
Unity's GC isn't even generational, a very bad mix with garbage heavy Clojure code. Expect confusing, bizarre and transient performance and memory use issues. Debugging will be a nightmare.
And on top of that, Stack Overflow doesn't have an unlimited supply of Clojure JVM answers; Clojure .Net through IL2CPP? Forget it; you'll have to figure everything out yourself.
It's generally a mistake to make the initial development more pleasant at the expense of long term maintainability; and I think that's the trade off on the table here.