The TCP FAIL metric is the same as `TCP AttemptFails` from `/proc/net/snmp`. It is the same value that you also get with `netstat -st` under `failed connection attempts`.
Typically a high TCP FAIL rate means, that some service on this host is trying to open a connection to a service that is not (or no longer) running. It could be an indicator of bad configuration (e.g. wrong IP addresses), firewall misconfiguration, or stopped services on other hosts (which leads to failed connection attempts on any host trying to access this stopped service).
The easiest way to find out what causes the failed connection attempts is to record and analyze a short period of traffic with tcpdump/wireshark. But this is out of scope for a monitoring tool like Instana.