vCenter Appliance - WebClient down with "Error: Service name is invalid"
Last week I had to shutdown my lab vCenter 6.7. As it's just a lab I just powered it off instead of shutting down. After a few days I powered it back on when I wanted it back, and it booted without issues. But then WebClient wouldn't come up. Just says page cannot be displayed.
I logged in to the appliance via SSH and tried to list service status
root@vcenter [ ~ ]# service-control --status --all
2019-04-04T10:43:41.796Z Error: Service name "applmgmt" is invalid.
2019-04-04T10:43:41.816Z Error: Service name "content-library" is invalid.
2019-04-04T10:43:41.837Z Error: Service name "vcha" is invalid.
2019-04-04T10:43:41.857Z Error: Service name "pschealth" is invalid.
2019-04-04T10:43:41.878Z Error: Service name "sps" is invalid.
2019-04-04T10:43:41.898Z Error: Service name "sca" is invalid.
2019-04-04T10:43:41.936Z Error: Service name "vmcam" is invalid.
2019-04-04T10:43:41.956Z Error: Service name "vmware-vpostgres" is invalid.
2019-04-04T10:43:41.976Z Error: Service name "vsphere-client" is invalid.
2019-04-04T10:43:41.997Z Error: Service name "vapi-endpoint" is invalid.
2019-04-04T10:43:42.017Z Error: Service name "vmware-postgres-archiver" is invalid.
2019-04-04T10:43:42.037Z Error: Service name "vsm" is invalid.
2019-04-04T10:43:42.057Z Error: Service name "updatemgr" is invalid.
2019-04-04T10:43:42.078Z Error: Service name "vmonapi" is invalid.
2019-04-04T10:43:42.098Z Error: Service name "vsan-health" is invalid.
2019-04-04T10:43:42.119Z Error: Service name "rbd" is invalid.
2019-04-04T10:43:42.156Z Error: Service name "vpxd-svcs" is invalid.
2019-04-04T10:43:42.176Z Error: Service name "statsmonitor" is invalid.
2019-04-04T10:43:42.264Z Error: Service name "imagebuilder" is invalid.
2019-04-04T10:43:42.284Z Error: Service name "vsan-dps" is invalid.
2019-04-04T10:43:42.304Z Error: Service name "cm" is invalid.
2019-04-04T10:43:42.342Z Error: Service name "mbcs" is invalid.
2019-04-04T10:43:42.362Z Error: Service name "vpxd" is invalid.
2019-04-04T10:43:42.382Z Error: Service name "netdumper" is invalid.
2019-04-04T10:43:42.403Z Error: Service name "rhttpproxy" is invalid.
2019-04-04T10:43:42.423Z Error: Service name "vsphere-ui" is invalid.
2019-04-04T10:43:42.444Z Error: Service name "eam" is invalid.
2019-04-04T10:43:42.464Z Error: Service name "perfcharts" is invalid.
2019-04-04T10:43:42.519Z Error: Service name "analytics" is invalid.
2019-04-04T10:43:42.539Z Error: Service name "cis-license" is invalid.
Running:
lwsmd vmafdd vmcad vmdird vmdnsd vmware-pod vmware-sts-idmd vmware-stsd vmware-vmon
Tried to start them using "service-control --start --all", but it didn't work.
I followed below steps to get it back running..
Make sure that you take a backup of your vCenter before trying this (Snapshot or Clone)
1) Stop all services
root@vcenter [ ~ ]# service-control --stop --all
2) Cd to the below dir
root@vcenter [ ~ ]# cd /storage/vmware-vmon/.svcStats
3) Create a new directory and move all .Jason files there
root@vcenter [ /storage/vmware-vmon/.svcStats ]# ls
stats_analytics.json stats_cm.json stats_netdumper.json stats_sps.json stats_vcha.json stats_vmware-postgres-archiver.json stats_vpxd-svcs.json stats_vsphere-client.json
stats_applmgmt.json stats_eam.json stats_perfcharts.json stats_statsmonitor.json stats_vdcs.json stats_vmware-sca.json stats_vsan-dps.json stats_vsphere-ui.json
stats_autodeploy.json stats_imagebuilder.json stats_pschealth.json stats_updatemgr.json stats_vmcam.json stats_vmware-vpostgres.json stats_vsan-health.json
stats_cis-license.json stats_mbcs.json stats_rhttpproxy.json stats_vapi-endpoint.json stats_vmonapi.json stats_vpxd.json stats_vsm.json
root@vcenter [ /storage/vmware-vmon/.svcStats ]# mkdir StatsjsonBKP
root@vcenter [ /storage/vmware-vmon/.svcStats ]# mv *.json /storage/vmware-vmon/.svcStats/mkdir StatsjsonBKP/
4) Do another ls to confirm
root@vcenter [ /storage/vmware-vmon/.svcStats ]# ls
mkdir StatsjsonBKP
5) Now it's time to start all services
root@vcenter [ ~ ]# service-control --start --all
Operation not cancellable. Please wait for it to finish...
Performing start operation on service lwsmd...
Successfully started service lwsmd
Performing start operation on service vmafdd...
Successfully started service vmafdd
Performing start operation on service vmdird...
Successfully started service vmdird
Performing start operation on service vmcad...
Successfully started service vmcad
Performing start operation on service vmware-sts-idmd...
Successfully started service vmware-sts-idmd
Performing start operation on service vmware-stsd...
Successfully started service vmware-stsd
Performing start operation on service vmdnsd...
Successfully started service vmdnsd
Performing start operation on profile: ALL...
Successfully started profile: ALL.
Performing start operation on service vmware-pod...
Performing start operation on service lwsmd...
Successfully started service lwsmd
Performing start operation on service vmafdd...
Successfully started service vmafdd
Performing start operation on service vmdird...
Successfully started service vmdird
Performing start operation on service vmcad...
Successfully started service vmcad
Performing start operation on service vmware-sts-idmd...
Successfully started service vmware-sts-idmd
Performing start operation on service vmware-stsd...
Successfully started service vmware-stsd
Performing start operation on service vmdnsd...
Successfully started service vmdnsd
Performing start operation on profile: ALL...
Successfully started profile: ALL.
Performing start operation on service vmware-pod...
Successfully started service vmware-pod
Bang!!
Gave few mins to start web client and all good!
