Genome Format: Even though I’m a long time programmer, I vastly preferred this year’s version where no one (except you) had to write any code. This was awesome!
Implementation/Spec: I would have preferred a clear spec accompanied by a reference implementation. Hy may be fun to use, but it’s incredibly slow. (Also the version differences causing various problems was no fun at all.)
The only big thing to watch out for is to not use the built-in RNG of whatever language you’ll end up using, but instead a relatively simple to implement (but still good quality) one like e.g. PCG where you can probably find a library, but if not then you have just a couple of lines to translate. (Different RNGs would mean different results. If everyone can use exactly the same, then all re-implementations are able to behave exactly the same, and it’s easy to test that you got your implementation correct.)
Time: If you can, avoid the very last/first week of a quarter year—that’s when companies and people who are self-employed have to finish all the bookkeeping. (I had several requests for clarification & planning the next quarter coming at me, that took away even more time...)
Duration: A week felt quite short, two weeks would have left a week to sort out infrastructure problems and then another one to solve the actual problem. Maybe next time you could split it in two, first week releases the spec and a sample environment (like the one that you used throughout the post), second week releases the actual parameters for the game. (I.e. the big 10-biome list and final feature costs.) Leaving a 48 hour window for changes (before submission opens) would be fine at that point; given that many would already understand the base setting, any quirks like grassland having less grass than rain forest would likely be noticed quickly and could still be fixed.)
Submission: Google forms sucked. Maybe there’s a browser extension that makes it easier to use, but I had to click around a lot and for some reason the browser built-in form completion was disabled for the fields, so I had to manually paste email / user etc. every. single. time. Ugh. There was also no way to see which ones I had already submitted. I’m pretty sure that I’m not the one with 11 species (and even if, two would have been exact duplicates), but an alternative “or just send me a CSV file formatted like the example data” would have been much more usable for me.
Still, in all it was a lot of fun and I’m curious what the results will be. Thank you very much for running another round this year, and I hope there will be another one next year!
Lots of great feedback! Here are my tentative thoughts. Please don’t construe them as promises.
Gene Format
The genome format is easier for me too. This sounds like the genome format is just the right way to go in the future.
Implementation/Spec
I plan to stick to Hy, but I’ll make the versioning clearer in the future. I could have done a better job with setup instructions overall. I think the slowness came from implementation choices. I made an extremely inefficient simulation in order to support forward compatibility with some features I never got around to implementing. I’m optimizing for features, not speed.
Using PCG sounds like an easy change. If that was all there was to it then I wouldn’t mind using it for the random numbers. However there’s other bits of the code too (like random selection from sets) which might vary from one implementation to another. Making it possible for everyone’s simulator to behave exactly the same is a nice-to-have feature that I’m probably not going to implement. I’d rather put that time into creating more adaptations players get the choice of using.
Time
I can avoid the last/first week of a quarter year. That’s no problem.
Duration
Extending from one week to two weeks is no problem either. Using the first week for a spec and a sample environment and the second week for the actual parameters sounds excellent. Leaving an explicit 48-hour window for changes sounds good too.
Submission
It’s easy for me to accept CSV files. I think the best way to help everyone is to just accept both CSV files and Google Forms.
I plan to stick to Hy, but I’ll make the versioning clearer in the future.
If there’s two weeks, that should leave enough time for making & checking alternate implementations, as well as clarifying any unclear parts. (I never fully understood the details of the selection algorithm (and it seems there were bugs in it until quite late), but given a week for focusing just on that, I hope that should work out alright.)
I’m optimizing for features, not speed.
No complaints here, that’s the only sane approach for research and other software like this.
However there’s other bits of the code too (like random selection from sets) which might vary from one implementation to another.
Compared to e.g. trying to get the Mersenne Twister (which I think is still Python’s default RNG?) either linked to or re-implemented in some other obscure language, that’s a trivial task. I don’t expect problems there, as long as those functions actually use the specified RNG and not internally refer to the language’s default.
Oh also, on genome formats: I’ve been doing quite a bit of stuff with domain specific languages / PLT stuff, as well as genetic algorithms and other swarm optimization approaches before. If you have any ideas that you want to bounce off someone or have some cool idea for some complex genome format that looks like it might be too much work, feel free to ping me—I might be able to help! (Ideally in a sufficiently abstract / altered way that I can still participate, but if need be, I’d give that up for a year too.)
Oh also, on genome formats: I’ve been doing quite a bit of stuff with domain specific languages / PLT stuff, as well as genetic algorithms and other swarm optimization approaches before. If you have any ideas that you want to bounce off someone or have some cool idea for some complex genome format that looks like it might be too much work, feel free to ping me—I might be able to help! (Ideally in a sufficiently abstract / altered way that I can still participate, but if need be, I’d give that up for a year too.)
Feedback on the game so far:
Genome Format: Even though I’m a long time programmer, I vastly preferred this year’s version where no one (except you) had to write any code. This was awesome!
Implementation/Spec: I would have preferred a clear spec accompanied by a reference implementation. Hy may be fun to use, but it’s incredibly slow. (Also the version differences causing various problems was no fun at all.)
The only big thing to watch out for is to not use the built-in RNG of whatever language you’ll end up using, but instead a relatively simple to implement (but still good quality) one like e.g. PCG where you can probably find a library, but if not then you have just a couple of lines to translate. (Different RNGs would mean different results. If everyone can use exactly the same, then all re-implementations are able to behave exactly the same, and it’s easy to test that you got your implementation correct.)
Time: If you can, avoid the very last/first week of a quarter year—that’s when companies and people who are self-employed have to finish all the bookkeeping. (I had several requests for clarification & planning the next quarter coming at me, that took away even more time...)
Duration: A week felt quite short, two weeks would have left a week to sort out infrastructure problems and then another one to solve the actual problem. Maybe next time you could split it in two, first week releases the spec and a sample environment (like the one that you used throughout the post), second week releases the actual parameters for the game. (I.e. the big 10-biome list and final feature costs.) Leaving a 48 hour window for changes (before submission opens) would be fine at that point; given that many would already understand the base setting, any quirks like grassland having less grass than rain forest would likely be noticed quickly and could still be fixed.)
Submission: Google forms sucked. Maybe there’s a browser extension that makes it easier to use, but I had to click around a lot and for some reason the browser built-in form completion was disabled for the fields, so I had to manually paste email / user etc. every. single. time. Ugh. There was also no way to see which ones I had already submitted. I’m pretty sure that I’m not the one with 11 species (and even if, two would have been exact duplicates), but an alternative “or just send me a CSV file formatted like the example data” would have been much more usable for me.
Still, in all it was a lot of fun and I’m curious what the results will be. Thank you very much for running another round this year, and I hope there will be another one next year!
Lots of great feedback! Here are my tentative thoughts. Please don’t construe them as promises.
Gene Format
The genome format is easier for me too. This sounds like the genome format is just the right way to go in the future.
Implementation/Spec
I plan to stick to Hy, but I’ll make the versioning clearer in the future. I could have done a better job with setup instructions overall. I think the slowness came from implementation choices. I made an extremely inefficient simulation in order to support forward compatibility with some features I never got around to implementing. I’m optimizing for features, not speed.
Using PCG sounds like an easy change. If that was all there was to it then I wouldn’t mind using it for the random numbers. However there’s other bits of the code too (like random selection from sets) which might vary from one implementation to another. Making it possible for everyone’s simulator to behave exactly the same is a nice-to-have feature that I’m probably not going to implement. I’d rather put that time into creating more adaptations players get the choice of using.
Time
I can avoid the last/first week of a quarter year. That’s no problem.
Duration
Extending from one week to two weeks is no problem either. Using the first week for a spec and a sample environment and the second week for the actual parameters sounds excellent. Leaving an explicit 48-hour window for changes sounds good too.
Submission
It’s easy for me to accept CSV files. I think the best way to help everyone is to just accept both CSV files and Google Forms.
If there’s two weeks, that should leave enough time for making & checking alternate implementations, as well as clarifying any unclear parts. (I never fully understood the details of the selection algorithm (and it seems there were bugs in it until quite late), but given a week for focusing just on that, I hope that should work out alright.)
No complaints here, that’s the only sane approach for research and other software like this.
Compared to e.g. trying to get the Mersenne Twister (which I think is still Python’s default RNG?) either linked to or re-implemented in some other obscure language, that’s a trivial task. I don’t expect problems there, as long as those functions actually use the specified RNG and not internally refer to the language’s default.
Oh also, on genome formats: I’ve been doing quite a bit of stuff with domain specific languages / PLT stuff, as well as genetic algorithms and other swarm optimization approaches before. If you have any ideas that you want to bounce off someone or have some cool idea for some complex genome format that looks like it might be too much work, feel free to ping me—I might be able to help! (Ideally in a sufficiently abstract / altered way that I can still participate, but if need be, I’d give that up for a year too.)
I love this forum/community so much.