Lately I read a number of articles describing how candidates can ace a job interview by using AI to get the answers. There are even You Tube videos showing such interviews. Selecting a capable candidate was always a tedious and somewhat risky activity. The ability of candidates to trick us by using AI increases the risk of recruiting mistakes in the future. How real is this concern?

I will get back to this, but let me digress.

I Wrote a Web App!

In the last few days, I developed a small, very simple web application in Python. I then added a few automated tests to validate it, again in Python, using Selenium and Chromedriver.

It took me about three days. For reasons that will soon become clear, I have no idea if three days are reasonable, exceptionally fast, or exceedingly slow progress.

Let me first give you some background about me, describe what I learned during this coding experience, and I’ll let you judge if this is an amazing or a dismal achievement.

My background

I have an Electronics Engineering degree. For the last 35 years I worked for Intel, first in a silicon manufacturing facility (Fab8 in Jerusalem), and for the last 25 years, until my recent retirement, I worked at Intel’s design center across the road. During my time in the fab I took classes in Computer Science and while I never got an official degree in CS, I did cover most of the topics in the CS curriculum not already covered by my EE degree.

In the design center I started testing stuff in the lab and worked my way up to Principal Engineer level, as a Software Validation Architect. I worked on the software that drove cable modems, Wi-Fi cards, smart TVs, graphic hardware, 3D cameras and AI-based computer vision applications. The projects I worked on were all embedded software. As a result, all during my career until my retirement, I never wrote a web application and never wrote any test automation code to test GUI or web applications.

One of the benefits of retirement is the freedom to spend your time as you wish. So, when my daughter came up with an idea for a simple web application, I decided to give it a try. Considering that my fee was much lower than the responses she got on Fiverr, I got the job (on probation :-)).

Correction: Microsoft Copilot Wrote a Web App

I got to work.

First, I wrote the requirements for the application, so I know what needs to be done. My daughter, who has a degree in CS, reviewed the requirements and agreed they are correct.

Because I don’t know enough Python and HTML to attempt such a project, I made extensive use of Microsoft’s Copilot, describing what I need, and mostly copy-pasting it into my application code. Which meant that most of the actual writing I did was prompts, rather than Python code.

Once the application was working, I used Copilot to write some automated tests in Python to check that my application will perform well when used by ~30 concurrent users. Copilot selected Selenium and Chromedriver for this task. I just needed to install the needed Python packages and the Chrome driver.

I lost about half a day trying to figure out how to deploy the app on Hostgator (the service I use to host my web site, www.testprincipia.com), until I found that my subscription does not include applications hosting. I moved to AWS, where I got it working in minutes, with the main obstacle being setting the correct access rules.

To summarize:

• Webapp: 2 days
• Testing: 0.5 day
• Hosting (including mistakes): 0.5 day

I am skipping the CSS work which took much more time than I anticipated – but that’s a different topic. For now, let’s stop at the point where I had a functional, admittedly not visually appealing, web app.
To me, three days looked an unbelievable short time. The perception is driven by knowing that had I attempted to create the same web app a few years ago, I’d be nowhere close to finishing after three days.
I’d need to take tutorials on:

• How to build a web application
• How to create an SQL database with Python
• How to automate with Selenium
• How to run the tests on multiple threads to simulate multiple users
• How to set hosting on AWS
  

I would have also needed to learn quite a lot about HTML, how to create pages that have buttons that link one page to another, how to pass parameters between pages, and of course something basic about CSS. Just to learn all this would have taken me three-four weeks, and then I’d use what I learned in many try and fail cycles of until I got the code to work.

Conclusion: With AI, we don’t need to learn much. AI can do the work for us.

Or is it so?

A look under the hood

While I was enjoying Copilot doing the heavy lifting for me, I also realized how none of the code it generated would make any sense to me, and would be totally useless to me, had I not accumulated a HUGE amount of knowledge during my thirty-five years in tech.

Here is a list of these knowledge items (not in any order; I am quite sure I am taking a lot of other knowledge for granted and missing some important stuff):

1. HTML
2. Databases; specifically: SQL databases and familiarity with SQLite
3. Linux
4. Git
5. Python
6. Code debugging
7. Visual Code and its extensions
8. Networking
9. Data structures

Some of the above topics are rather simple; some take a long time to master – even at the basic level. Acquiring familiarity and confidence in using them takes more time. Not less important than knowing how to use these tools and concepts, is the knowledge about these tools, concepts and systems. Understanding what they are good for and when they apply.

In theory, if I asked Copilot about any of the above, I am sure it would have helped me a lot. But that’s the catch: I need to be aware of things like HTML, SQL, Python virtual environment, to ask Copilot to help me with them. Had I had no clue about any of these, I don’t think I’d have made much progress.
The image below shows (at a very high level) when I started accumulating the knowledge that allowed me to interact effectively with Copilot.

Each item in this list was acquired through many hours of use and struggle with the new knowledge. Some activities (such as requirements engineering) were my main job for some years. The results of these efforts were deep familiarity and long-term retention of the knowledge. Naturally this was not ALL I learned during these years… I just marked the items relevant to the web-app project.

(Infographics based on Slidesgo and Freepik)

You can argue I had a lousy learning curve; that I should have learned more, faster. I don’t argue with that. But I learned things as they were needed to get my work done. If my role was building web applications with Python, I am certain I’d have learned most of these in the first year. Maybe the first few months.

Takeaways

Here are two conclusions I had following my experience.

a) AI is an amazing power multiplier to anyone writing code. It allows us to get more done, even in technologies and languages we are unfamiliar with.

b) To enjoy the benefits, one must acquire a solid basis in many aspects of programming and technology.

I see this as an encouraging result. On one hand I can tackle challenges that were practically infeasible for me before AI, and on the other hand, my hard-earned experience counts. I can’t be replaced by just anyone with a pulse.

But what about those interviews?

My experience shows that when candidates can ace an interview using AI, it means they know A LOT and have a very healthy technological background.

In short: If we ask open questions – those who are not phrased as something you can just drop to the prompt (e.g. “write a function in python that can do this”) – we should be able to recognize candidates who have solid background. And these are the people we should be happy to recruit – especially when they manage to do this in real time. If they phrase intelligent prompts and can talk intelligently about the AI output, they are not fakes. They will likely be effective, both in the stuff they know already and in harnessing AI to help them do things they are not familiar with (yet). In fact, we should encourage candidates to use AI, share that second screen with us and talk through how they are going about answering our interview questions.

I’d like to share a story I heard from a friend—let’s call her Jane—about an experience she had during her army service. Jane served at a shop where defective equipment was fixed and returned to service. The shop was using various tools as well as a good supply of paintbrushes. According to the procedures, at the end of each workday all tools had to be returned to their proper places and all paintbrushes had to be cleaned. In reality, at the end of the workday the soldiers were tired or in rush to get somewhere, and the shop was left a mess.

Of course, by the next day the paintbrushes would be stiff with dry paint and had to be thrown away. As new work was coming in regularly, there was never time to tidy the shop. Expensive tools were lost and precious time was spent either looking for them or going to the quartermaster to get new ones.

Jane testifies that she acted this way, too, and did not really see any problem in such behavior: “This is how things were done there,” she said.

A few months before her service time was up, Jane found herself in a unique position. All the senior soldiers in her shop had completed their service term and were released. She was the only one left to train a new group of soldiers on how to do the work and what their responsibilities were.

Jane made a decision to do something drastic. A day before the new group arrived, she cleaned and tidied the shop until it was sparkling, with everything in its proper place. When the new soldiers came in, she explained the training plan and goals. She strongly emphasized a number of times, “The most important thing in this shop is neatness. We never go home before the shop is squeaky clean, all tools are in their proper places, and all paintbrushes are cleaned up.”

Every training task given to the new soldiers contained a section about neatness. If any of the soldiers failed to clean up or misplaced a tool, he or she was reprimanded and had to listen to a long sermon about the indispensable importance of tidiness.

The result? By the end of the training period—about four months—the shop had a new generation of soldiers who were fanatic about neatness, and it did not even occur to them that things could be done any other way. A new tradition was created: keeping the rules and working according to the defined procedures.

Overcoming Resistance to Change

When we try to bring a quality process into a development or test team, there tends to be resistance. People say the process slows down progress or blocks innovation, that too much discipline does not fit a development team, or that the new procedure makes the workplace feel like a production facility. I believe this attitude is mostly the result of tradition—people being too used to the organizational culture and work habits to think that there could be a better way to do things.

Assume for a minute that you left your job and joined a new company. In that company, people adhere to processes and procedure and everyone, without exception, follows them to the letter. As a newcomer, you immediately follow the example set by everyone else. Because everybody keeps the rules, you won’t feel like a pushover for following the rules; you won’t think you are wasting time or that it feels like a factory line. You will just internalize the idea that these are the rules of the game, and it will feel natural to perform them.

Understanding this influence can help in making successful process changes in your team.

Implementing a New Tradition

Making process and procedural changes using a new set of rules and behaviors is an effort to create a new tradition. Considering that no one plans to fire everybody in your company and get an entirely new generation of workers (I hope), you have to get people to adopt these different work habits. Still, don’t expect the logic of these implementations alone to get people on board and make them want to change. You have to make these new processes a habit—a tradition—and this can only be achieved if everyone does them all the time.

This is not impossible, as long as management and the team support it. Management has to commit that the new rules will not be tossed out the minute there is any inconvenience or other pressures. And the team needs to decide on the minimal level of process that is needed and pledge to each other to follow this process, no matter what. Agree that it’s all right to remark when someone does not follow the rules and to demand compliance from each other.

For inspiration, watch Mars Code, a lecture by Gerard Holzmann, senior research scientist at NASA’s Jet Propulsion Laboratory that describes how its group of developers adopted new quality processes. Their model definitely can be reused with other software teams, and it gives a great example of how a new work tradition can be built.

After a while, behaviors that the team believes are important will start to take root. Eventually it will not even feel too hard to do them. And there’s another bonus: New people who join the team won’t even have to be convinced that the rules are important or justified. It will just be “how things are done here.”

Published in Hebrew on “Testing World” magazine (3^rd issue, October 2015)
http://itcb.org.il/index.php?option=com_k2&view=item&id=759:מגזין-עולם-הבדיקות-גיליון-שלישי-רבעון-4-2015&Itemid=865   

Published in English: June 6, 2016
https://www.stickyminds.com/article/secret-change-management-creating-new-tradition

It’s a classic dispute: Two test automation engineers can’t agree on which programming language to use. In some contexts the strong points of a certain language definitively make it the right choice, but what do you do when either language could work well for a project? That’s when it becomes a managerial decision.

It was years ago, while working as a chip-level test engineer, that I first I witnessed an argument that has repeated itself a dozen times since. Two test automation engineers could not agree which programming language was better for their next-generation automation system: Pascal or C? (I did mention that it was years ago, right?)

Each side had weighty arguments: what’s easier to do in each language, how important strong string-processing is, what language has better libraries, etc. Additionally, both sides claimed that there isn’t really a great need to align on one language—it is possible to write modules in one language and use them as libraries for the other language.

The arguments continued for a while. If my memory serves, the automation was mostly written in C for a simple reason: It was the preferred language of the developer who wrote most of the code.

There are cases where the decision is clear-cut and easy, because in some contexts the strong points of a certain programming language definitively make it the right choice. But what do you do when this is not the case, and each language has a list of pros and cons?

Years later I was consulting for a team of testers writing a software test automation framework. Except for the fact that the languages discussed were now Visual Basic and C#, the rest was very familiar: the endless arguments; the teleconferences and face-to-face meetings, vowing to continue until we reach a decision; the presentation of proofs by each side about why their position is the correct one—all while employing both languages as each side tries to generate enough code to weigh the decision so that their language is chosen.

Who is right?

It seems to me that everyone is right … and no one is right.

The claims on each side are usually correct, which means you can’t settle the argument based purely on technology attributes. Comparing the pros of one language to those of the other is really an argument about the reasons for both languages to exist. Clearly each language has some aspects where it is better than the other. When you compare the positive and negative aspects of any language, it is hard to judge which parameter is more important. With no single strong argument that tilts the scales in favor of one or the other, the discussions drag on an on.

In these situations, I think it’s important to realize that the main consideration should be the number of languages, not which language.

Picking a Language Is a Managerial Decision

There is an “activation energy” for each new language used by a team. When context-switching from one language to the other, the engineers lose some time recalling the technical details of the language. You may need to duplicate some of your development tool (e.g., IDE, code control) and support systems, such as your nightly build and continuous integration. The engineers will need to learn how to work and be proficient with more tools. All this takes time, which costs money.

Engineers already are not very keen on reusing someone else’s code. There is always the feeling of “It will take me less time to just rewrite the whole thing from scratch.” The chance of code reuse when that code is not written in the language the engineer prefers approaches that of pigs flying.

When you need to transfer ownership of code between engineers due to a career move, role change, or the proverbial bus, you will find that the language used to write the code narrows the list of possible candidates. You will have a harder time recruiting as well, as you will be looking for candidates that know both languages fairly well. Alternatively, you will need to pay for the new hire’s learning curve.

The conclusion is that in many cases, the decision on a programming language is not an engineering decision, but rather a managerial decision.

The manager should listen to both sides to see if indeed this is a situation with no clear argument for one side. If that is the case, the parameters to consider are managerial:

• What support tools does the organization use?
• What language is known by more engineers?
• What language is the tested code written in?
• What’s the preferred language of the person who will write most of the code?
• What’s the language used for more of the existing code?
• What language is used by systems that need to interface with your code, and will you need to update those systems?

Of course, there also will be other, situation-dependent (nontechnical!) factors.

Note how these questions do not try to decide if one language is technologically superior to the other; they sidestep that issue entirely. These are objective questions that in most cases are easy to answer. If any of the answers does not help in the decision—such as, there is no one language that significantly more engineers are familiar with—you can just drop this aspect from the list.

But what if, even after evaluating the above questions, there isn’t a clear choice? At this point, it doesn’t matter which language you choose; the important thing is that you do choose one, because the delay may be more expensive than the cost incurred if you made the wrong decision. As the band Rush once sang so well, “If you choose not to decide, you still have made a choice.”

So flip a coin, get the team to back whatever the outcome is, and follow through with supporting that language.

Published in Hebrew on “Testing World” magazine (2nd issue, July 2015, p.7)

Published in English: December 28, 2015  http://www.stickyminds.com/article/less-more-picking-your-test-automation-language