ChatGPT should NOT be trusted to give you production ready code

It should be treated just like any other tool

This is probably obvious to anyone who has been programming for any length of time, but you should not expect ChatGPT to give you production ready code!

Some of you are probably going to be like “Mike, what is with you and your hate for ChatGPT?!” Nah fam, I don’t hate ChatGPT. I’m skeptical of it, namely because of all the hype around it…

The hype

Right away, within just a couple months of it coming out, I’ve seen some of y’all jump head over heels for this product. Some of you, have even hit me up acting like you have discovered something that’ll upend some mature established businesses overnight! All because you asked it to write some code to do some useful stuff for you (e.g scrape a web page for emails) and it did.

I remember some of the wild speculation in my suggested articles list about “coding is over”, “ChatGPT can do my job”, …

I seen what seemed to be no end of developers trying to cram ChatGPT (or something similar) into their products, to hell with the consequences…

I even heard of one rich CEO making the controversial, and imo dumbass decision to just use ChatGPT instead of hiring developers…

I seen low-code, and even no-code app development solutions being pushed…

I even seen one of my favorite coding YouTube channels Fireship pushing their share of hype about ChatGPT/AI

Of course, I’ve seen people swear by using it in place of well…Googling… WTF.html

Let’s get real here…

ChatGPT is an advanced text generator. That’s it.

It is NOT Google (or a replacement thereof).

It is NOT a lawyer.

It is NOT a developer replacement.

It. Generates. Text (or code)!

It is only as “smart” as the content that it has been trained on (by users). and the internet is full of wrong answers!

Unlike a Google search, you cannot filter/sort its answers, let alone by relevance or date. It’s pretty much a yes-man.

It is prone to “hallucinations”; it’ll sometimes make up library methods that don’t exist (e.g. how to get a WebElement from a TestObject in Katalon Studio)

It is not a replacement for expert, especially software developer

It, if you use it right, will supplement your coding skills.

It can help you with API/library/framework methods of which you may have little or no familiarity. For example, it helped me with handling token response errors in Gmail API for Java.

It can help you with build tool configuration files (e.g. Gradle).

It can help you with terminal scripting (e.g. Git BASH)

It can help you with writing util methods.

Sometimes it may fail at the question you ask, but if you ask it a question that’s a couple of steps smaller, it can answer that one…and it may guide you to writing the code yourself. For example, when I ask it for a generic deep copy method…

You should still be skeptical of its output

As I mentioned prior, it can hallucinate library methods that don’t exist.

It can give you non-working code…There’s a reason it’s banned from StackOverflow

Its coding abilities, according to OpenAI’s own whitepaper
rank firmly in the category of newbie.

Also, as the title of this post suggests…it is NOT going to give you production-ready code!

What do you mean “production-ready code”?!

When I say “production-ready code”, I mean code that is free from code smells. Learn more about those here.
I mean code that is :

• intuitive. If you have to rely on documentation just to tell you what a line of code does, you’re doing it wrong.

• separation of concerns

• this hits on code smell, but code that doesn’t repeat itself. This is the Don’t-Repeat-Yourself (DRY) principle

The quality of ChatGPT’s code output, leaves much to be desired. Yes, the code may work, however, more times than not:

• it loves to cram everything in one place, typically one file, one class, even one method…

• because of that, there may be little separation of concerns, and even

• lots of code repetition instead of creating util method.

For example, instead of including, for example, a build.gradle file to download dependencies to the Gradle project, it’ll use Grape to download the dependency directly from the Groovy script file that needs it!

Let’s review some sample code

As for those other bullet points, let’s look at a code snippet it gave me for sending a Google Calendar invite in Java :

import java.io.IOException;
import java.security.GeneralSecurityException;
import java.util.Arrays;
import java.util.Date;
import java.util.TimeZone;
import java.util.List;

import com.google.api.client.auth.oauth2.Credential;
import com.google.api.client.googleapis.auth.oauth2.GoogleCredential;
import com.google.api.client.util.DateTime;
import com.google.api.services.calendar.Calendar;
import com.google.api.services.calendar.CalendarScopes;
import com.google.api.services.calendar.model.Event;
import com.google.api.services.calendar.model.EventAttendee;
import com.google.api.services.calendar.model.EventDateTime;

public class GoogleCalendarUtil {
  
  private static final String APPLICATION_NAME = "Your Application Name";
  private static final List<String> SCOPES = Arrays.asList(CalendarScopes.CALENDAR_EVENTS);
  private static final String SERVICE_ACCOUNT_EMAIL = "your-service-account-email@your-project-id.iam.gserviceaccount.com";
  private static final String KEY_FILE_LOCATION = "path/to/your/key.p12";
  private static final String TIME_ZONE = "America/Los_Angeles";

  /**
   * Send a Google Calendar invite to a user email with specified event details.
   * 
   * @param userEmail the email address of the user to whom the invite is sent
   * @param eventTitle the title of the event
   * @param eventDescription the description of the event
   * @param eventStart the start time of the event
   * @param eventEnd the end time of the event
   * @throws IOException
   * @throws GeneralSecurityException
   */
  public static void sendGoogleCalendarInvite(String userEmail, String eventTitle, String eventDescription, Date eventStart, Date eventEnd) throws IOException, GeneralSecurityException {
    Credential credential = authorize();
    Calendar service = new Calendar.Builder(GoogleCredential.getTransport(), GoogleCredential.getJsonFactory(), credential)
        .setApplicationName(APPLICATION_NAME)
        .build();

    Event event = new Event()
        .setSummary(eventTitle)
        .setDescription(eventDescription);

    DateTime startDateTime = new DateTime(eventStart, TimeZone.getTimeZone(TIME_ZONE));
    EventDateTime start = new EventDateTime().setDateTime(startDateTime);
    event.setStart(start);

    DateTime endDateTime = new DateTime(eventEnd, TimeZone.getTimeZone(TIME_ZONE));
    EventDateTime end = new EventDateTime().setDateTime(endDateTime);
    event.setEnd(end);

    EventAttendee[] attendees = new EventAttendee[] {
      new EventAttendee().setEmail(userEmail),
    };
    event.setAttendees(Arrays.asList(attendees));

    event = service.events().insert("primary", event).execute();
    System.out.printf("Event created: %s\n", event.getHtmlLink());
  }

  /**
   * Authorize the Google Calendar API client.
   * 
   * @return the authorized credential
   * @throws IOException
   * @throws GeneralSecurityException
   */
  private static Credential authorize() throws IOException, GeneralSecurityException {
    GoogleCredential credential = new GoogleCredential.Builder()
        .setTransport(GoogleCredential.getTransport())
        .setJsonFactory(GoogleCredential.getJsonFactory())
        .setServiceAccountId(SERVICE_ACCOUNT_EMAIL)
        .setServiceAccountPrivateKeyFromP12File(new java.io.File(KEY_FILE_LOCATION))
        .setServiceAccountScopes(SCOPES)
        .build();
    return credential;
  }
}

Look at how much sendGoogleCalendarInvite() is having to take on here!

• setting up the Calendar service

• creating an EventDateTime to start the event

• creating an EventDateTime to end the event

• creating the Collection of EventAttendees

• creating the Event from all that, and finally

• hitting the API to post this event

:o

Somehow it was worse the first time I asked ChatGPT for this code… authorization code was in that method the first time I asked it!

Definitely we could move this to util methods, and even create a data model for the event. This data model should be what we pass the main method `sendGoogleCalendarInvite()`…

I won’t spell out how on here, just want to highlight obvious code quality issues with raw ChatGPT output…I asked it to DRY up the code, for example, and it ran out of tokens trying.

Now imagine if you plugged all that in your code base as-is, and you either wanted to tweak it or fix issues you were facing with it… You’d have a bad time, with all those concerns happening in one method! That AI would be your crutch!

…and this brings me to conclusion:

AI is a tool, and should be used as such

Just like any other tool (e.g. StackOverflow forum, Google search), you should use it with a healthy amount of skepticism.

Try what it gives you, but not blindly.

Definitely refactor anything it gives you to conform to sound coding principles, and the design decisions of your project.

Definitely learn how to code, and make sure you fully understand what ChatGPT gives you, otherwise you end up becoming a slave to the tool.

I say this as a user, and as someone whose line of work literally is telling computers what to do.

AI is a TOOL! Use it wisely!!