Always add a newline after printing the hostname: entry, just in case
userdata API call returns *just* the hostname.
Update the test data to just such a bare hostname.
Fixes#59
In case networking takes as long as a minute to come up, give hostname
lookup up to 100 seconds to complete. Give initial HTTP request up to 120
seconds _from when we started_ to succeed, so even if we spent all our
time waiting on DNS, we still give 20 more seconds for the HTTP server.
To improve package test capabilities, introduce a test target that will
attempt to fetch SSH keys and user data from a locally-spawned test
server. Add other bits in ucd-data-fetch.c to support testing.
Add fetch_test, which spawns an HTTP server to serve some sample
user-data and cloud-config files, and verify that ucd-data-fetch can
retrieve them.
Additional URI https://metadata.platformequinix.com/userdata
Also add missing users key in generated equinix conf
Our output for the equinix cloud config file omitted the "users:" key
prior to defining the user.
Don't close the socket until we're done reading.
Make sure the socket fd/stream gets closed in all the error paths
preceding our close/fclose.
Also ensure the output file is closed properly in error cases.
To provision on equinix we can grab the `keys` file from their
service. The address I entered in the template is tentative - it
needs to be checked that metadata.platformequinix.com resolves to that
address in their internal network as well that it's on a static IP
address and not some RR load balancer service - ideally.
Some servers require the host name to be sent in the user-data API request.
We can allow a host name instead of IP address in the config struct by
doing a lookup if it's not an IP address already.
The data fetcher tool operated assuming the key file presented on the
server contains only 1 key. However, several cloud providers use this
file to present multiple keys to the client systems at provisioning time
and this should be functional.
Without this change, only the first key will be properly inserted into
yaml, and all secondary keys will result in a yaml failure or be ignored
entirely. It would look something like this:
```
ssh_authorized_keys:
- ssh-rsa <correct key 1>
ssh-rsa <this key 2 will be ignored>
ssh-rsa <this key 3 will be ignored>
```
The template is modified to allow line-by-line reading of the key file
from the server, and each line written will be prefixed with the yaml
" - " entry prefix in the output file.
When writing the SSH keys to our generated cloud-config file, make sure
to follow with an extra line feed before we concatenate the user-data
contents. Otherwise, we could end up including the first line of the
user-data response in the authorized_keys file, e.g.
ssh-rsa <...key...> user@host#cloud-config
^^^^^^^^^^^^^
Instead, worst case, /var/lib/cloud/<provider>-user-data might just have
an extra (ignored) blank line.
There are some cases where a config-drive is on the system, but
not initially detected early at boot. One such case is when the
cdrom subsystem isn't builtin to the kernel but enabled as a module.
We can try and catch this issue. It will slow down boot a bit, so
this only waits 1 second total for now. But modprobing `sr_mod` which
depends on `cdrom` is a good start here, and covers obtaining the
`meda_data.json` file in case a VM is booted with the -native or
other non-kvm kernels.
These lines forced systemd-networkd to wait until `ucd` had
completed.
In turn, `ucd` requires the network to be active in order
to install packages. It waits for the network to become
active, even.
Surely, this is untenable. Ucd will have to accept any
network state and may not prevent it from starting as
early as possible, which would be counter to any fast
boot initiative.
This option allows users to suspend execution of the cloud-init provisioning
until a valid network is detected.
This detection is currently done through requesting a DNS lookup for one
of the Clear Linux NTP service IP addresses (this is a RR record, we
don't actually look at the result).
This lookup is not infinite. After 5 minutes, the wait exits no matter
what even if there is no network detected.
The option can be provided manually. `packages` and `package_upgrade`
options *imply* this option, but one can explicitly disable the wait
by providing it early in the cloud-config file with a value of `false`.
The wait routing is active - it will retry relatively quickly to detect
an active connection. Any failure will result in another retry. In
a fully private network without public DNS, this will not work.
The DNS hostname used for testing can be manipulated through the
`-with-dnstestaddr=<hostname>` configure flag. You shouldn't put an
IP address in here, since that fully disables any network testing.
When I deduplicated this code I mistakenly used parse_headers() in
the same way twice. This was incorrect.
We definitely need to error out if a 404 or something else happens
for the SSH keys, since then we've lost entirely.
But a missing `user-data` is mostly harmless and optional, so, in
case a 404 happens for the second `GET` request, we shouldn't error
out, because this makes the instance unusable. Instead, we can
carry on as normal.
Robert Dower