Lately I read a number of articles describing how candidates can ace a job interview by using AI to get the answers. There are even You Tube videos showing such interviews. Selecting a capable candidate was always a tedious and somewhat risky activity. The ability of candidates to trick us by using AI increases the risk of recruiting mistakes in the future. How real is this concern?
I will get back to this, but let me digress.
I Wrote a Web App!
In the last few days, I developed a small, very simple web application in Python. I then added a few automated tests to validate it, again in Python, using Selenium and Chromedriver.
It took me about three days. For reasons that will soon become clear, I have no idea if three days are reasonable, exceptionally fast, or exceedingly slow progress.
Let me first give you some background about me, describe what I learned during this coding experience, and I’ll let you judge if this is an amazing or a dismal achievement.
My background
I have an Electronics Engineering degree. For the last 35 years I worked for Intel, first in a silicon manufacturing facility (Fab8 in Jerusalem), and for the last 25 years, until my recent retirement, I worked at Intel’s design center across the road. During my time in the fab I took classes in Computer Science and while I never got an official degree in CS, I did cover most of the topics in the CS curriculum not already covered by my EE degree.
In the design center I started testing stuff in the lab and worked my way up to Principal Engineer level, as a Software Validation Architect. I worked on the software that drove cable modems, Wi-Fi cards, smart TVs, graphic hardware, 3D cameras and AI-based computer vision applications. The projects I worked on were all embedded software. As a result, all during my career until my retirement, I never wrote a web application and never wrote any test automation code to test GUI or web applications.
One of the benefits of retirement is the freedom to spend your time as you wish. So, when my daughter came up with an idea for a simple web application, I decided to give it a try. Considering that my fee was much lower than the responses she got on Fiverr, I got the job (on probation :-)).
Correction: Microsoft Copilot Wrote a Web App
I got to work.
First, I wrote the requirements for the application, so I know what needs to be done. My daughter, who has a degree in CS, reviewed the requirements and agreed they are correct.
Because I don’t know enough Python and HTML to attempt such a project, I made extensive use of Microsoft’s Copilot, describing what I need, and mostly copy-pasting it into my application code. Which meant that most of the actual writing I did was prompts, rather than Python code.
Once the application was working, I used Copilot to write some automated tests in Python to check that my application will perform well when used by ~30 concurrent users. Copilot selected Selenium and Chromedriver for this task. I just needed to install the needed Python packages and the Chrome driver.
I lost about half a day trying to figure out how to deploy the app on Hostgator (the service I use to host my web site, www.testprincipia.com), until I found that my subscription does not include applications hosting. I moved to AWS, where I got it working in minutes, with the main obstacle being setting the correct access rules.
To summarize:
- Webapp: 2 days
- Testing: 0.5 day
- Hosting (including mistakes): 0.5 day
I am skipping the CSS work which took much more time than I anticipated – but that’s a different topic. For now, let’s stop at the point where I had a functional, admittedly not visually appealing, web app.
To me, three days looked an unbelievable short time. The perception is driven by knowing that had I attempted to create the same web app a few years ago, I’d be nowhere close to finishing after three days.
I’d need to take tutorials on:
- How to build a web application
- How to create an SQL database with Python
- How to automate with Selenium
- How to run the tests on multiple threads to simulate multiple users
- How to set hosting on AWS
I would have also needed to learn quite a lot about HTML, how to create pages that have buttons that link one page to another, how to pass parameters between pages, and of course something basic about CSS. Just to learn all this would have taken me three-four weeks, and then I’d use what I learned in many try and fail cycles of until I got the code to work.
Conclusion: With AI, we don’t need to learn much. AI can do the work for us.
Or is it so?
A look under the hood
While I was enjoying Copilot doing the heavy lifting for me, I also realized how none of the code it generated would make any sense to me, and would be totally useless to me, had I not accumulated a HUGE amount of knowledge during my thirty-five years in tech.
Here is a list of these knowledge items (not in any order; I am quite sure I am taking a lot of other knowledge for granted and missing some important stuff):
- HTML
- Databases; specifically: SQL databases and familiarity with SQLite
- Linux
- Git
- Python
- Code debugging
- Visual Code and its extensions
- Networking
- Data structures
Some of the above topics are rather simple; some take a long time to master – even at the basic level. Acquiring familiarity and confidence in using them takes more time. Not less important than knowing how to use these tools and concepts, is the knowledge about these tools, concepts and systems. Understanding what they are good for and when they apply.
In theory, if I asked Copilot about any of the above, I am sure it would have helped me a lot. But that’s the catch: I need to be aware of things like HTML, SQL, Python virtual environment, to ask Copilot to help me with them. Had I had no clue about any of these, I don’t think I’d have made much progress.
The image below shows (at a very high level) when I started accumulating the knowledge that allowed me to interact effectively with Copilot.
Each item in this list was acquired through many hours of use and struggle with the new knowledge. Some activities (such as requirements engineering) were my main job for some years. The results of these efforts were deep familiarity and long-term retention of the knowledge. Naturally this was not ALL I learned during these years… I just marked the items relevant to the web-app project.
(Infographics based on Slidesgo and Freepik)
You can argue I had a lousy learning curve; that I should have learned more, faster. I don’t argue with that. But I learned things as they were needed to get my work done. If my role was building web applications with Python, I am certain I’d have learned most of these in the first year. Maybe the first few months.
Takeaways
Here are two conclusions I had following my experience.
a) AI is an amazing power multiplier to anyone writing code. It allows us to get more done, even in technologies and languages we are unfamiliar with.
b) To enjoy the benefits, one must acquire a solid basis in many aspects of programming and technology.
I see this as an encouraging result. On one hand I can tackle challenges that were practically infeasible for me before AI, and on the other hand, my hard-earned experience counts. I can’t be replaced by just anyone with a pulse.
But what about those interviews?
My experience shows that when candidates can ace an interview using AI, it means they know A LOT and have a very healthy technological background.
In short: If we ask open questions – those who are not phrased as something you can just drop to the prompt (e.g. “write a function in python that can do this”) – we should be able to recognize candidates who have solid background. And these are the people we should be happy to recruit – especially when they manage to do this in real time. If they phrase intelligent prompts and can talk intelligently about the AI output, they are not fakes. They will likely be effective, both in the stuff they know already and in harnessing AI to help them do things they are not familiar with (yet). In fact, we should encourage candidates to use AI, share that second screen with us and talk through how they are going about answering our interview questions.