Implement your own command line interface in golang

In this article, we will build a command line interface (CLI) together with a real-world example: Scraping a webpage. A CLI is providing a straightforward method for executing tasks, configuring settings, and managing services. Therefore implementing your own CLI in a programming language like Go (Golang) can be incredibly beneficial for several reasons:

Image source: Cobra | Viper

1. Can encapsulate complex operations into simple commands (Encapsulation - Clean Code). Enhances the likelihood of acceptance and adoption among team members by providing a user-friendly interface to complex systems.
2. Can serve as the cornerstone of standardized operational workflows within a team
3. Can enforce predefined protocols and sequences of actions that need to be followed, thereby minimizing deviations and potential errors.

or just build a CLI because GUIs are so mainstream, and you're not about that life—you're here to type like a hacker in a movie :)

Let's start

For building the CLI, we are using the beautiful libraries Cobra and Viper, which enhance command structure and configuration management in Go. Check out their GitHub pages to explore their features further, and if you find them useful, consider leaving a star!

By the way, the well-known kubectl command-line tool of Kubernetes is also built with these libraries.

• Framework for building CLI tools - Cobra
• Configuration for setting up the CLI - Viper
• Cobra-cli: Managing CLI applications built with Cobra - Cobra-CLI

The complete code can be found in the My-First-Cli repository. At the end of the article, I'll showcase a more complex scenario, including scraping the index.html page of a website.

Set up

Im using go version 1.22 - go version. Let’s start by setting up your CLI project properly. First, you'll need to create a new Go project and install the Cobra CLI tool. Follow these detailed steps to get everything up and running:

# Create folder
mkdir my-first-cli
cd my-first-cli

# Install cobra-cli
go install github.com/spf13/cobra-cli@latest

# Create go project
go mod init github.com/raphaeldeveloperberlin/my-first-cli

# Install dependencies
go get -u github.com/spf13/viper
go get -u github.com/spf13/cobra@latest

# Create cobra cli application
cobra-cli init --viper

# Check if all depndencies were downloaded
go mod tidy

• Command not found: cobra-cli. Check end of article.

The folder structure should now appear as follows:

my-first-cli/
│
├── cmd/
│   └── root.go      # Viper config & cobra root command
│
├── go.mod           # Dependency management
├── go.sum           # Dependency management
├── LICENSE          # Define your License
└── main.go          # Main go programm

Viper configuration

By default, Viper reads the configuration from your home directory - $HOME/.my-first-cli.yaml. I prefer to have the config in the root directory of the go project, just as a personal preference.

To accommodate this, we need to make a few small changes in the Viper configuration (root.go).

+ rootCmd.PersistentFlags().StringVar(&cfgFile, "config", "", "config file (default is $PWD/.my-first-cli.yaml)")
- rootCmd.PersistentFlags().StringVar(&cfgFile, "config", "", "config file (default is $HOME/.my-first-cli.yaml)")

+ actualDirectory, err := GetActualDirectory()
- home, err := os.UserHomeDir()

+ viper.AddConfigPath(actualDirectory)
- viper.AddConfigPath(home)


# Add following function to the end of the file
func GetActualDirectory() (string, error) {
	originalDir, err := os.Getwd()
	if err != nil {
		return "", err
	}
	return originalDir, nil
}

Now we can create the .my-first-cli.yaml file in the root folder my-first-cli and Viper will automatically read it as the default configuration. If you prefer to change the configuration location at the time of command execution, there is a predefined --config flag available for this purpose.

Create the first command

For creating the first command, execute the following command:

# In root folder (my-first-cli)
cobra-cli add scrape

Feel free to modify the Short and Long descriptions inside the scrape file to better clarify the use case.

For example:

+ Short: "Scraping the index.html file from a website."
- Short: "A brief description of your command",

Lets try it

Let's go ahead and try it out:

# In root folder (my-first-cli)
# Quick testing
go run .

# Binary execution
go build .

# Test it
./my-first-cli

By default, when no specific command is provided, Cobra automatically displays all the available options of the CLI right out of the box. We can see our nearly created scrape command as well. That’s amazing! So lets try the scrape command.

# Quick testing
go run . scrape

# Binary execution
go build .
./my-first-cli scrape

# Result
scrape called

Setting up a command is quite straightforward, but let's walk through a real example to address more complex aspects such as flags and configuration.

Example

The example involves scraping a website and storing the HTML at a predefined location. Therefore, we need one flag for the website URL. The location information will be obtained from the configuration file (.my-first-lcoation.yaml).

Explaining the code for scraping the website and storing it to disk would exceed the scope of the article. However, feel free to contact me on GitHub and ask any questions you may have.

Create flag website-url

To create the flag, we add the following statement to the init function in scrape.go. Here we define the name of the flag first, followed by no default value, and finally a short description.

scrapeCmd.Flags().String("website-url", "", "Select a website-url")

To validate the flag as required, we check if the flag is set. We retrieve the website URL value from the command, and if it is empty, we throw an error and halt the program. Add the function to the end of the scrape.go file.

// Add to import "github.com/spf13/cobra"

func ValidateFlagWebsiteUrl(cmd *cobra.Command) string {
	websiteUrlFlag, err := cmd.Flags().GetString("website-url")
	if err != nil {
		log.Fatalf("Exception while reading flags - %v", err)
	}
	if websiteUrlFlag == "" {
		log.Println("Please add the flag website-url")
		os.Exit(1)
	}
	return websiteUrlFlag
}

Create location in configuration

Inside the .my-first-cli.yaml we create a new key-pair. Of course, adjust the path according to your computer's filesystem.

location: /path/to/your/location/index.html

To validate the location as required, we check if the location is set. Add the function to the end of the scrape.go file.

// Add to import "github.com/spf13/viper"

func ValidateLocationConfig() string {
	location := viper.GetViper().GetString("location")
	if location == "" {
		log.Println("Please add the location to .my-first-cli.yaml")
		os.Exit(1)
	}
	return location
}

Defined the scrape command:

Here, we integrate all the functions together and execute the main program. Adjust the following command block in scrape.go as follows:

var scrapeCmd = &cobra.Command{
	Use:   "scrape",
	Short: "Scraping the index.html file from a website.",
	Long: `Scraping the index.html from a website involves retrieving the main HTML...`,
	Run: func(cmd *cobra.Command, args []string) {
		websiteUrl := ValidateFlagWebsiteUrl(cmd)
		location := ValidateLocationConfig()
		log.Println("Start to scrape website %v to location %v", websiteUrl, location)
		scrapeAndExtractHtml(websiteUrl, location) // Implementation next block
	},
}

For the main program, the chromedp library is used. Please ensure it is downloaded.

# Add to import "github.com/chromedp/chromedp"
go get github.com/chromedp/chromedp

Lastly, let's integrate all this other stuff to the end of the scrape.go file. Its the logic for scraping and saving the html page to disk. The code isn't structured. For a larger project consider organizing it into separate packages to improve clarity and maintainability.

func scrapeAndExtractHtml(websiteUrl, location string) {
	ctx, cancel := chromedp.NewContext(context.Background())
	defer cancel()
	ctx, cancel = context.WithTimeout(ctx, 20*time.Second)
	defer cancel()

	var html string

	GetHtmlFromMainDom(websiteUrl, ctx, &html)
	saveToDisk(location, html)
}

func GetHtmlFromMainDom(webSiteUrl string, ctx context.Context, html *string) {
	if err := chromedp.Run(ctx,
		chromedp.Navigate(webSiteUrl),
		chromedp.WaitVisible(`html`, chromedp.ByQuery),
		chromedp.OuterHTML("html", html, chromedp.ByQuery),
	); err != nil {
		log.Fatal(err)
	}
}

func saveToDisk(filepath, filecontent string) {
	buffer := createBufferForHtmlString(filecontent)
	bufferToDestination(buffer, filepath)
}

func createBufferForHtmlString(filecontent string) *strings.Builder {
	buffer := new(strings.Builder)
	buffer.WriteString(filecontent + "\n")
	return buffer
}

func bufferToDestination(codeToCopyBuffer *strings.Builder, filepath string) {
	CreateFoldersIfNotExist(filepath)
	writeResultToDisk(codeToCopyBuffer, filepath)
}

func CreateFoldersIfNotExist(destinationFile string) {
	destinationFolder := filepath.Dir(destinationFile)
	err := os.MkdirAll(destinationFolder, os.ModePerm)
	if err != nil {
		log.Println("Error creating destination folder:", err)
		return
	}
}

func writeResultToDisk(toSaveContent *strings.Builder, pathFilename string) {

	openFile, err := os.Create(pathFilename)
	if err != nil {
		log.Print("File couldn't be created or truncated.")
	}
	defer func() {
		if err := openFile.Close(); err != nil {
			panic(err)
		}
	}()

	r := strings.NewReader(toSaveContent.String())
	w := bufio.NewWriter(openFile)

	buf := make([]byte, 1024)
	for {
		n, err := r.Read(buf)
		if err != nil && err != io.EOF {
			panic(err)
		}
		if n == 0 {
			break
		}

		if _, err := w.Write(buf[:n]); err != nil {
			panic(err)
		}
	}

	if err = w.Flush(); err != nil {
		panic(err)
	}
}

Final Result

Let's give it a try! First we build the cli.

go build .

# Test it
./my-first-cli scrape --website-url https://example.com

The index.html file has been successfully downloaded into the predefined location.

There are further topics like structuring commands and adding boolean flags etc. Feel free to customize your CLI according to your specific use cases. Have fun exploring!

Errors

Helper for possible errors.

Command not found: cobra-cli

# should show cobra-cli
ls $GOPATH/bin

# Check if GOPATH/bin is set in PATH variable
export GOPATH=$(go env GOPATH)
export PATH=$PATH:$GOPATH/bin