Setup Julia Repo in GitHub with Actions

Install Julia

Basically if you are using Linux or Mac, you can easily install Julia by the package manager. For Gentoo, we can install it with portage as following.

$ sudo emerge -av julia

The problem to install Julia by Gentoo package manager is slow upgrade speed. In this time, July 27 2023, latest stable version version of Julia in the official site is 1.9.2 but Gentoo still has 1.8.5 as the latest one. To overcome this gap, according to the instruction in Julia GitHub we can easily install it by compiling source.

$ cd localgit  <-- local git directory
$ git clone https://github.com/JuliaLang/julia.git
$ cd julia
$ git checkout v1.9.2  <-- tag for latest version
$ make
$ ln -s julia ~/.local/bin/julia  <-- This should be earlier path than /usr/bin/julia

Setup Julia language server with formatter for Neovim

To setup LSP(Language Server Protocol) with julials.lua file in Neovim-from-scratch, we need to setup nvim-lspconfig environment first. Using stow tool, we can copy Makefile to ~/.julia/environments/nvim-lspconfig/.

$ cd localgit/evoagile_configs   <-- if your local cloned repo is in localgit
$ stow -t ~ julia
$ cd ~/.julia/environments/nvim-lspconfig/
$ make

After make command, neovim is launched and open Example.jl source file with some errors. If you wait a moment and no problem occur to install, you can find some bullets as following screenshot.

Then just quit neovim and continue and finish installation. To test installation, open any example Julia file and change keyword function to func tion and at the same line, input gl. If you have some bullets and messages as following, your installation is successful.

If you have any error to load Julia LSP, you can find following errror message in neovim.

As you can find in the following, LanguageServer package depends on JuliaFormatter, StaticLint, and SymbolServer. That means we do not have to install JuliaFormatter separately.

using Pkg
Pkg.add("LanguageServer")

    Updating registry at `~/.julia/registries/General.toml`
   Resolving package versions...
  No Changes to `~/work/JuliaStudy.jl/JuliaStudy.jl/docs/Project.toml`
  No Changes to `~/work/JuliaStudy.jl/JuliaStudy.jl/docs/Manifest.toml`

To format your code, you can find related commends in keymaps.lua file. Type :Format in command or <space key>f(when <localleader> = <space key>) in normal mode, whole code in the file will be formatted.

To define format parameters in Julia code is not easy in LSP configuration file. The solution is use '.JuliaFormatter.toml' file in every repo. So it can be convenient to add this file to PkgTemplates. (See below section)

Use PkgTemplates to generate repository

PkgTemplates simplifies the process of setting up the initial file structure for a new repository, specifically CI integration with GitHub actions. To use PkgTemplates, we recommend to install it in system default Julia environment, for example, v1.9 as following.

using Pkg
Pkg.activate()   # activate system default env
Pkg.add("PkgTemplates")

After installing it, we need to setup the template. It would be convenient to keep it in the startup.jl file for easy reuse. Because this file runs every time Julia starts up, we can utilize the generate function as shown in the following example.

function genGithubRepo(userName::String, repoName::String)
  templateGithub = Template(;
    user = userName,
    dir = "~/localgit",
    julia = v"1",  # for [compat] section in Project.toml
    plugins = [
      # Use semantic version, See Julia Pattern book page 43.
      ProjectFile(; version = v"1.0.0-DEV"),
      License(; name = "MIT", path = nothing, destination = "LICENSE"),
      Git(;
        branch = LibGit2.getconfig("init.defaultBranch", "master"),
        ssh = true,
        jl = true,
        manifest = false),
      GitHubActions(;
        destination = "CI.yml",
        linux = true,
        osx = false,
        windows = false,
        x64 = true,
        x86 = false,
        coverage = true,
        extra_versions = ["1.8", "1.9", "nightly"]),
      CompatHelper(; destination = "CompatHelper.yml", cron = "0 0 * * *"),
      TagBot(;
        destination = "TagBot.yml",
        trigger = "JuliaTagBot",
        token = Secret("GITHUB_TOKEN"),
        ssh = Secret("DOCUMENTER_KEY"),
        ssh_password = nothing,
        changelog = nothing,
        changelog_ignore = nothing,
        gpg = nothing,
        gpg_password = nothing,
        registry = nothing,
        branches = nothing,
        dispatch = nothing,
        dispatch_delay = nothing),
      Codecov(),
      Documenter{GitHubActions}(logo = Logo(;
        light = homedir() * "/.config/logo/logo.png",
        dark = homedir() * "/.config/logo/logo-dark.png")),
      Dependabot(),
    ])
  generate(templateGithub, repoName)
end

genGithubRepo("ecoinfos", "JuliaStudy.jl")

To run above code, folowwing things are needed.

• Logo files in $HOME/.config/logo. For example, use stow command to my evoagile_configs repository to setup evoagile logos as following.

$ cd localgit/evoagile_configs
$ stow -t ~ logo

• DOCUMENTER_KEY. This is required for deploying documents and can be generated by DocumeterTools.

• CompatHelper access permission to pull request. Go to GitHub organization(Ex. ecoinfos) -> settings -> Actions -> General and set options with Read and write permissions and Allow GitHub Action to create and approve pull requests. After this organization setting, we can setup repository setting as the same way. That is, without organization setting, we cannot change permission in repository setting. Without this options, you can get error messages as like this.

{
    "message":"GitHub Actions is not permitted to create or approve pull requests.",
    "documentation_url":
    "[https://docs.github.com/rest/pulls/pulls#create-a-pull-request]
    (https://docs.github.com/rest/pulls/pulls#create-a-pull-request)"}""",
    HTTP.Exceptions.StatusError(403, "POST",
    ...
}

Use GitHub-CLI to create repository in GitHub

Because PkgTemplates only make repository in localgit folder we need to register it in GitHub. Basically this can be possible in GitHub Web page but GitHub-CLI tool make this more easy. In Gentoo, we can install it as following.

$ sudo emerge -av github-cli
$ gh auth login

After some authentication step, we can access, create and sync an example repository as following.

$ gh repo list "ecoinfos"   # check the list of repo in ecoinfos user.
$ gh repo create "ecoinfos/JuliaStudy.jl" --public

# This repo is required to publish document to GitHub.
$ gh repo create "ecoinfos/ecoinfos.github.io" --public

$ cd localgit/JuliaStudy
$ git push origin HEAD --tags

Playing with REPL

To setup REPL, andreypopp/julia-repl-vim is installed in newvim (already installed in Neovim-from-scratch) and Julia default environment as following.

$ julia    # install in default env.
julia> using Pkg; Pkg.add(url="https://github.com/andreypopp/julia-repl-vim")

Because the startup.jl file contains required startup commands, we can use it without manual setup.

using REPLVim
@async REPLVim.serve()

Once REPL is launched in another terminal and open a Julia file, we can connect the code to REPL by :JuliaREPLConnect or <localleader>o.

• <localleader>o: Connect current file to REPL. You need to connect every files to REPL if you want to send them.
• <localleader>u: In normal mode, send the cursor line to REPL. In visual mode, send all lines in the range to REPL.
• <localleader>k: In visual mode, send exact range to REPL. This is useful if you want to send a part of code in a line to REPL.
• <localleader>/: Format and send code whose block has same indented level.

Coding with snippets

To make coding more easy, a few snippets are defined in julia.lua. To add and understand the definition of them you can refer to offical LuaSnip site. Some frequently used snippets are

• def : Define function. Type df at the beginning of a line, this automatically generates function end block. There are three fields to input and when finish one, you can shift to next field by <F2> key with ignoring any possible suggestions.
• ds : Define structure. Type ds at the beginning of a line, this generate struct end block. You also can find -> mark in the screen and that means there are choices to select. Type <Ctrol + h> or <Ctrol + l> you can change the selection between struct and mutable struct. After selection one, type <F2> to go next field.

For more snippets, please refer to julia.lua.

Run tests and check documents

When you finish to setup environment and you can try to run all tests in JuliaStudy repository as following.

$ cd ~/localgit/JuliaStudy
$ julia --project=.
julia> using Pkg
julia> Pkg.test()
...
Test Summary: | Pass  Broken  Total     Time
All Tests     |  316       1    317  4m53.5s
     Testing JuliaStudy tests passed

This is important to run all tests before you commit your codes. If you have no error messages as above result, you can submit your documents in Github. The Broken test is skipped test for temporary exception.

To check the documents for Documenter tool, run make.jl in docs project as following.

$ cd ~/localgit/JuliaStudy/docs
$ julia --porject=.
julia> include("make.jl")

If you have an error as following,

julia> include("make.jl")
ERROR: LoadError: ArgumentError: Package JuliaStudy [d399ac94-384f-4751-bea9-3a8f3e663572]
is required but does not seem to be installed:
 - Run `Pkg.instantiate()` to install all recorded dependencies.
...

you need to add JuliaStudy as a development environment.

julia> Pkg.develop(path="..")

Setup Julia Mono Nerd font with input method in Neovim

Julia uses unicodes extensively for mathematical symbols. For example in chapter 9 of Julia Patters book, "∘" is used for composed operator as following.

julia> top_story_id = first ∘ fetch_top_stories

To support unicode fully, I recommend to use Julia Mono font with Nerd patch. You can find full list of unicode in Julia Unicode Input page. As this page shows, to input unicode like "∘" in REPL, input \circ with tab key. To support these kinds of shortcuts in Neovim, I defined the Lua snippets in the file, lur/user/LuaSnip/julia.lua as following code.

local math_symbs = {
	{ name = "alpha", symbol = "α" },
	{ name = "beta", symbol = "β" },
	{ name = "gamma", symbol = "γ" },
	{ name = "delta", symbol = "δ" },
	{ name = "Delta", symbol = "Δ" },
	{ name = "lambda", symbol = "λ" },
	{ name = "Lambda", symbol = "Λ" },
	{ name = "theta", symbol = "θ" },
	{ name = "mu", symbol = "μ" },
	{ name = "pi", symbol = "π" },
	{ name = "eta", symbol = "η" },
	{ name = "varphi", symbol = "φ" },
	{ name = "psi", symbol = "ψ" },
	{ name = "phi", symbol = "ϕ" },
	{ name = "epsilon", symbol = "ϵ" },
	{ name = "sigma", symbol = "σ" },
	{ name = "circ", symbol = "∘" },
    ...
}

So, for more unicode shortcuts in Noevim, we can add additional symbols with shortcuts in the file. To get the matching symbol with unicode point or search symbol by name, refer to following Julia code.

using Glyphy
glyphy(0x02218)

02218   ∘✓  ring operator ⌨ \circ

glyphy("ring operator")

02218   ∘✓  ring operator ⌨ \circ
0229a   ⊚✓  circled ring operator ⌨ \circledcirc
 found 2 glyphs matching "ring operator"

To do or practice

1. Add '.JuliaFormatter.toml' file to PkgTemplates to define format option.
2. Try to understand GitHub security model with secret keys. See SSH Deploy Keys Walkthrough and security hardening for github actions.

References

1. Github repo: evoagile_configs
2. Github repo: Neovim-from-scratch/27-Julia-REPL
3. Setting up Julia lsp for Neovim
4. discourse: Neovim + LanguageServer.jl
5. Hands-On Design Patterns and Best Practices with Julia