I saw some job posts online that require you to take a tech art evaluation exam. Can anyone give me some guidance on what they consist of? Or example questions?
Really? Can you point me to a few, I’m curious!
We used to have TA tests as part of the interview process here. Write a scripted plugin in Max that does such-n-such, write a script to place N objects randomly on the surface of another object, etc…
Cool, haven’t heard of anyone doing that.
Think I’ve only been asked to do an Art Test once, needless to say I passed on the ‘opportunity’!
Man I love taking tests…
We have a TA test here that involves taking an exist .NET class we have in our library, and subclassing it to fulfill a certain requirement. We then ask for some additional functionality, add-ons, and classes. We’ve had a number of people take it and gotten results all over the map with a great distribution- one perfect, one entirely wrong, most on the spectrum in the middle), which I think is good. I’ve also found it maps pretty well to how competent and well-adjusted the coders here are, too- and it’s possible to come up with even better ways than the ‘answer key’ test.
Ultimately a good test will be able to give an idea of:
- Your skills. Do you have the basic skills required for getting the job done? This is the easiest way to throw people out who aren’t up to the task, but we tend to go easy on stylistic choices or problems, even moreso on newbies or people moving from native->managed code.
- Your critical thinking. Any good test has all the info needed to get it done. How well are you able to execute what needs doing to the asked for specs? What sort of answers or results do you give? (ie, sometimes people will have ‘trick’ questions that demand an answer that questions the question, often times on low-level implementation details (“How many classes can be in a namespace in .NET” is one I read recently). Critical thinking is necessary to ‘pass’ the test. You can write code that does what was asked but the code is designed completely wrong, proper critical thinking means your thought process is coherent and solid.
- How well can you infer? Any good test has some implicit requirements or assumptions- in our example, it is a class with very little documentation that you must extend. You really need to understand the code in order to know how best to go about what you need to do. Someone who is able to ‘gel’ with the inferences and nuances involved in the test and codebase is an ace.
Ultimately tests are valuable in judging your skills and how you adapt to problems. And unfortunately, there’s little you can do to prepare, unlike a standardized test. There are few good examples and people usually don’t share them (a good test can last a long time). If we create a new TA test here, I’ll be sure to post the old one.
Sounds pretty cool Rob. Definitely will check it out if you ever put it online.
Personally though, I’m not a big fan of tests. I’ve seen people who’ve passed tests but proved to be tragic in production and others who struggled with tests or refused to do one who I know are extremely valuable to have and perform beyond all expectations.
Then again that’s why talent acquisition is really a complicated task!
The only place I know that does ‘tests’ in film industry is Dreamworks. Although their tests are less hands on and more theory in linear algebra and the likes.
Rob, thanks for the info.
@ShadowM8 thats what the interview is really for. However evaluating how someone works on a team via having them come in and be in the spotlight for long periods of time answering the same questions from many different people is difficult. The book Rework has some interesting business ideas including reshaping the interview process. But I digress, this thread is about tests.
Our tests are simple, yet seek the same information that Rob pointed out. One of our tests is python/xml heavy, just data mining and reporting kind of stuff. Another is more practical in regards to integration into engines. We set up a model that is fairly broken and ask people to fix it. Another is to rig a critter. We’ll often combine all these tests into one larger one. It all depends on the exact need we have at the time.
I’m not sure how I feel about tests… Especially the ‘on-the-spot’ variety.
We give a test to candidates before they are invited to the office for an interview to asses their skills; a series problems increasing in difficulty. We give the candidates 48 hours to do as many as they can as thoroughly as they can but make clear that we don’t expect them to finish them all, and that thorough answers but not finishing is vastly better than bad answers for every problem…
Doing on-site tests, especially with no resources (internet, etc), is a gross over-expectation of a candidate, imho. I’ve been in the industry for 15 years, and I have to look up MaxScript reference and .Net methods constantly. If I’m programming, and I DON’T have MSDN, code project examples, or whatever open in a browser on my second monitor, I’m know I’m probably doing something wrong. (Hell, I have to look up all the formatting of the MacroScript heading every time I write one…LOL!) I’d certainly never expect a candidate to be able to code much of anything purely from memory.
Our on-site questions are almost exclusively of the white-board variety where we’d just like to get you to think out-loud and use VERY light pseudo-code, etc. This give an excellent indication of the candidates ability to communicate his thinking process rather than just demonstrate that he can or cannot do something…
This is the process for Tech artists and programmers alike…
just my $02.
Regarding having references available when coding. I know this opinion is unpopular, but putting someone under that sort of condition can be an important indication of ‘expert’ skills. I wouldn’t hire someone as a senior MXS or senior .NET developer if they couldn’t write something I expect them to know how to do without reference. Obviously I don’t care about syntax issues, but I don’t expect someone to have to look up sample code to create a scripted plugin in MXS, or use custom events in .NET. Having a quick and large memory is important to being a really good developer- when people come to you with questions, or you offer feedback, you aren’t going to be as effective if you use google as a crutch.
On the other hand, tests are often asking the applicant to implement some sort of unfamiliar concept, in which case we are doing sort of the opposite and seeing how well he can process and adapt the information available.
I just see it as a popular sentiment now to beat up on questions or tests where people don’t get references (and to some extent, compilers). Someone who relies on references is, for the most part, not going to be as fast, strong, or experienced as someone who is able to internalize and remember the information. I would consider it as a factor when looking for a senior-level developer.
I think depriving people of reference material to check their ‘expert’ skills is misguided at best. I’ve never worked with a tech artist or software engineer who programmed from memory. I myself reference my own code constantly, for starters, and not to toot my own horn but I develop rather fast!
Reminds me of the first in person interview out of college. Had the lead AI programmer come in and asked me to write the Dijkstra pathfinding algorithm on the whiteboard with correct syntax. Best interview ever :laugh:
And how’d you do? :):
[QUOTE=ShadowM8;7193]I think depriving people of reference material to check their ‘expert’ skills is misguided at best. I’ve never worked with a tech artist or software engineer who programmed from memory. I myself reference my own code constantly, for starters, and not to toot my own horn but I develop rather fast![/QUOTE]
I wouldn’t call it misguided, I’d call it guided in a different direction. Like I said, I’m sure I’m in the minority. I obviously have different experiences and look for different traits.
Really? Can you point me to a few, I’m curious!