Add support for optimized C compilation to .syso objects.

It took me a while to figure out why optimized C compilation seemed
to generate miscomputation of the Fibonacci number sequence. It appears
to be an unresolved issue with Go's internal linking which is discussed
here:

  https://github.com/golang/go/issues/24321

For a compute kernel, it seems important to be able to accommodate
compiler optimization. This adds some refinement for the strategy
I'm exploring to address:

  https://bugzilla.kernel.org/show_bug.cgi?id=216610

Signed-off-by: Andrew G. Morgan <morgan@kernel.org>

3 files changed, 104 insertions, 26 deletions

diff --git a/contrib/bug216610/Makefile b/contrib/bug216610/Makefile
index c83284c..aca7ab9 100644
--- a/contrib/bug216610/Makefile
+++ b/contrib/bug216610/Makefile
@@ -5,15 +5,15 @@ GOTARGET=$(shell eval $$(go env) ; echo $${GOHOSTOS}_$${GOARCH})
 
 all: go/fib
 
-go/fib: go/main.go go/vendor/fibber/fib.syso go/vendor/fibber/fib.go go/vendor/fibber/fibs_linux_amd64.s go/vendor/kernel.org/pub/linux/libs/security/libcap/psx
+go/fib: go/main.go go/vendor/fibber/fib.go go/vendor/fibber/fibs_$(GOTARGET).s go/vendor/fibber/fib_$(GOTARGET).syso go/vendor/kernel.org/pub/linux/libs/security/libcap/psx
 	cd go && CGO_ENABLED=0 go build -o fib main.go
 
 go/vendor/kernel.org/pub/linux/libs/security/libcap/psx:
 	mkdir -p go/vendor/kernel.org/pub/linux/libs/security/libcap/
 	ln -s $(topdir)/psx $@
 
-go/vendor/fibber/fib.syso: c/fib.c
-	gcc -c -o go/vendor/fibber/fib_$(GOTARGET).syso c/fib.c
+go/vendor/fibber/fib_$(GOTARGET).syso: c/fib.c ./gcc_$(GOTARGET).sh
+	./gcc_$(GOTARGET).sh -O3 c/fib.c -c -o go/vendor/fibber/fib_$(GOTARGET).syso
 
 clean:
 	rm -f *~
diff --git a/contrib/bug216610/README.md b/contrib/bug216610/README.md
index ae59a0d..a05a828 100644
--- a/contrib/bug216610/README.md
+++ b/contrib/bug216610/README.md
@@ -12,34 +12,50 @@ filed](https://bugzilla.kernel.org/show_bug.cgi?id=216610) against the
 [`"psx"`](https://pkg.go.dev/kernel.org/pub/linux/libs/security/libcap/psx)
 package. The bug-filer was seeking an alternative to `CGO_ENABLED=1`
 compilation needing the `cgo` variant of `psx` build. However, the go
-`"runtime"` package will
+`"runtime"` package will always
 [`panic()`](https://cs.opensource.google/go/go/+/refs/tags/go1.19.2:src/runtime/os_linux.go;l=717-720)
-if you try this.
+if you try this because it needs `libpthread` and `[g]libc` to work.
 
-However, in researching that bug, I have learned there is a trick to
-combining a non-CGO built binary with compiled C code. I learned about
-it from a brief reference in the [Go Programming Language
+In researching that bug report, however, I have learned there is a
+trick to combining a non-CGO built binary with compiled C code. I
+learned about it from a brief reference in the [Go Programming
+Language
 Wiki](https://zchee.github.io/golang-wiki/GcToolchainTricks/).
 
-This present directory evolved from my attempt to understand and
+This preset directory evolved from my attempt to understand and
 hopefully resolve what was going on as reported in that bug into an
-example of this _trick_.
+example of this _trick_. I was unable to resolve the problem as
+reported because of the aformentioned `panic` in the Go
+runtime. However, I was able to demonstrate embedding C code in a Go
+binary without use of cgo. So, a Go-native version of `"psx"` is thus
+achievable. This is what the example in this present directory does.
 
-*Caveat Emptor*: this example is potentially very fragile. The Go team
-only supports `cgo` linking against C.
+*Caveat Emptor*: this example is very fragile. The Go team only
+supports `cgo` linking against C. That being said, I'd certainly like
+to receive bug fixes, etc for this directory if you find you need to
+evolve it to make it work for your use case.
 
 ## Content
 
 In this example we have:
-- Some C code, `fib_init()` and `fib_next()` that combine to implement
-a _compute engine_ to determine [Fibonacci
+
+- Some C code for the functions `fib_init()` and `fib_next()` that
+combine to implement a _compute engine_ to determine [Fibonacci
 Numbers](https://en.wikipedia.org/wiki/Fibonacci_number). The source
-for this is in the sub directory `.../c/fib.c`.
-- Some Go code, in the directory `.../go/vendor/fibber` that uses this
+for this is in the sub directory `./c/fib.c`.
+
+- Some Go code, in the directory `./go/vendor/fibber` that uses this
 C compiled compute kernel.
+
+- `gcc_linux_amd64.sh` which is a wrapper for `gcc` that adjusts the
+compilation to be digestible by Go's (internal) linker. Using `gcc`
+directly instead of this wrapper generates an incomplete binary -
+which miscomputes the expected answers. See the discussion below for
+what might be going on.
+
 - A top level `Makefile` to build it all.
 
-This build uses vendored Go packages so I could experiment with
+This build uses vendored Go packages so one can experiment with
 modifications of the `"psx"` package to explore potential changes (of
 which there have been none).
 
@@ -53,7 +69,7 @@ $ make all
 $ cd contrib/bug216610
 $ make clean all
 ```
-When you run `.../go/fib` it should generate the following output:
+When you run `./go/fib` it should generate the following output:
 ```
 $ ./go/fib
 psx syscall result: PID=<nnnnn>
@@ -72,26 +88,32 @@ the reported [Bug
 this example offers an alternative path to build a `nocgo` that links
 to compute engine style C code.
 
+The reason we have added the `./gcc_linux_amd64.sh` wrapper for `gcc`
+is that we've found the Go linker has a hard time digesting the
+cross-sectional `%rip` based data addressing that various optimization
+modes of gcc like to use. Specifically, if a `R_X86_64_PC32`
+relocation entry made in a `.text` section is intended to map into a
+`.rodata.cst8` section in a generated `.syso` file, the Go linker
+seems to replace this reference with a `0` offset to `(%rip)`. What
+our wrapper script does is rewrite the generated assembly to store
+these data references to the `.text` section. The Go linker has no
+problem with this _same section_ relative addressing.
+
 ## Future thoughts
 
 At present, this example only works on Linux with `x86_64` (in
 go-speak that is `linux_amd64`). This is because I have only provided
 some bridging assembly for Go to C calling conventions on that
-architecture target (`.../go/vendor/fibber/fibs_linux_amd64.s`).
+architecture target (`./go/vendor/fibber/fibs_linux_amd64.s`).
 
 Perhaps a later version will have bridging code for all the Go
 supported Linux architectures, but it will also have to provide some
-mechanism to build the `.../c/fib.c` code to make
+mechanism to build the `./c/fib.c` code to make
 `fib_linux_<arch>.syso` files. The [cited
 bug](https://bugzilla.kernel.org/show_bug.cgi?id=216610) includes some
 pointers for how to use Docker to support this.
 
-The compilation optimization level for `.../c/fib.c` seems to be
-important for this example. Depending on which version of the compiler
-is being used, the optimization process can make more or less use of
-link-time optimizations, which don't seem to work in this example. For
-this reason, we don't include `-O<n>` gcc options when compiling that
-C file.
+## Reporting bugs
 
 Please report issues or offer improvements to this example via the
 [Fully Capable `libcap`](https://sites.google.com/site/fullycapable/)
diff --git a/contrib/bug216610/gcc_linux_amd64.sh b/contrib/bug216610/gcc_linux_amd64.sh
new file mode 100755
index 0000000..0f7fe83
--- /dev/null
+++ b/contrib/bug216610/gcc_linux_amd64.sh
@@ -0,0 +1,56 @@
+#!/bin/bash
+#
+# The Go linker does not seem to know what to do with relative
+# addressing of rodata.* offset from %rip. GCC likes to use this
+# addressing mode on this architecture, so we quickly run into
+# mis-computation when the relative addressing used in a .syso file of
+# symbol located data is resolved to completely the wrong place by the
+# Go (internal) linker.
+#
+# As a workaround for this, we can modify the assembly source code
+# generated by GCC to not point at problematic '.rodata.*' sections,
+# and place this data in the good old '.text' section where Go's
+# linker can make sense of it.
+#
+# This script exists to generate a '.syso' file from some '*.c' files.
+# It works by recognizing the '*.c' command line arguments and
+# converting them into fixed-up '*.s' files. It then performs the
+# compilation for the collection of the '*.s' files. Upon success, it
+# purges the intermediate '*.s' files.
+#
+# The fragile aspect of this present script is which compiler
+# arguments should be used for the compilation from '.c' -> '.s'
+# files. What we do is accumulate arguments until we encounter our
+# first '*.c' file and use those to perform the '.c' -> '.o'
+# compilation. We build up a complete command line for gcc
+# substituting '.s' files for '.c' files in the original command
+# line. Then with the new command line assembled we invoke gcc with
+# those. If that works, we remove all of the intermediate '.s' files.
+setup=0
+args=()
+final=()
+ses=()
+for arg in "$@"; do
+    if [[ "${arg##*.}" = "c" ]]; then
+	setup=1
+	s="${arg%.*}.s"
+	"gcc" "${args[@]}" -S -o "${s}" "${arg}"
+	sed -i -e 's/.*\.rodata\..*/\t.text/' "${s}"
+	final+=("${s}")
+	ses+=("${s}")
+    else
+	if [[ $setup -eq 0 ]]; then
+	    args+=("${arg}")
+	fi
+	final+=("${arg}")
+    fi
+done
+echo final: "${final[@]}"
+echo args: "${args[@]}"
+echo ses: "${ses[@]}"
+"gcc" "${final[@]}"
+if [[ $? -ne 0 ]]; then
+    echo "failed to compile"
+    exit 1
+fi
+rm -f "${ses[@]}"