Thank you in advance
It depends on what type of caregiving your business intends to do. The things you mentioned would apply to starting almost any business. You mentioned insurance - lots of potential liability issues. You'd need to be bonded and licensed. You'd need a caregiver hiring / vetting process. You'd need a training program. You'd need some company policies. You'd need a HCM / payroll system/service. You'd need an Federal EIN. You'd need a CRM system/service. You'd need a caregiver logging system/service of some sort (for liability and continuity of service as caregivers get swapped in and out). You'd need a scheduling system/service. You'd need a 24x7 contact system - ideally part of the CRM system - assuming caregivers stay over night.
There's a lot of good information on the internet. This is what I know just from dealing with caregiver organizations for my father...