@oripessach - The two regressions you've mentioned are both tied to OpenFL. We're stuck behind a rock and a hard place, needing to update OpenFL/Lime to take in fixes on their end and having to put up with some regressions along the way. We emphasize in a way, because now, we're the customers of OpenFL reporting bugs and waiting on fixes. It all comes full circle.
There are a few ways to deal with upstream changes and regressions, all of them less than ideal. I don't know how vocal you are when dealing with the OpenFL project, but I would expect someone in your position to advocate for your users and be at least somewhat active in trying to get the maintainers of upstream projects to address issues.
You could fork the OpenFL code and backport certain fixes for a more controlled release cycle of Stencyl.
On your end, having clearly defined beta releases can help you to deal with the uncertainty involved in pulling huge changesets from OpenFL into Stencyl.
I've had to deal with large open source projects breaking things for my day job, and sometimes there's no way to avoid just diving into the code and figuring out how to fix something that the upstream maintainers broke. It's not my favorite thing to do, but it's part of the job. In other cases I had to be pretty active in pushing the maintainers to fix specific issues, and follow up to make sure that it got done. Sometimes all it takes to get something fixed is to complain about it and to explain the impact an issue has on your users - no developer enjoys it when their users have a bad experience.
Unit testing has constantly been on our minds, but it's challenging to do this in a meaningful way with a complex UI app. Basic tests will catch the easy stuff, but there are so many possible actions, especially in the block editor, that designing a good suite tests for that seems daunting. Given your 30 years of experience, how do apps like ours tackle this challenge?
I'm not a fan of unit testing. I think it's a waste of time for anything but the most stable pieces of infrastructure code - file systems, encryption libraries, media codecs and such. For UI projects you need to do something else.
Automating a quick smoke test verifying that you can create a game, import a couple of resources, create a scene and a couple of behaviors and compile a game should be relatively manageable using any number of tools designed to test Java applications. You don't have to test everything, but testing some basic functionality will at least prevent the most embarrassing bugs from slipping through. I believe there was at least one release that couldn't compile to Flash, for example. You should be able to catch that sort of thing, at least. Or the latest regression where some blocks can't be snapped into place.
I don't have experience testing Java GUI apps, but a quick search shows that there's a few open source tools out there that are designed to do that. You might want to see if any of them could do the trick.
As for volunteer work, it isn't so much the issue of incentives as it is drudge work that not a lot of people like to commit to in the long term. We used to have volunteer-based QA, but the people doing it couldn't keep up with it in the long term. The best thing we can do is to clamp down harder and ask ourselves to more rigorously test what we check in, but we've all been guilty at some point of checking in code with minimal testing. It's just like documentation, which many devs don't enjoy doing.
Yeah, QA is hard work. Good QA folks like to get paid for their work... There's probably no way around that. And you're right - staying disciplined about testing your own code is extremely important. I wish I could say I've never checked in untested changes, but I can't. What I can say is that I'm typically paranoid about releasing something to production that has embarrassing bugs, so I tend to spend some time writing automated testing tools and running extensive functional test suites overnight before I let any big release hit the wire. It's an approach that worked well for me.
Show Posts
