Remove index.php from page URL

My website is working fine except the url can also be accessed by adding index.php like https://www.mywebsite.in/index.php/about-us/contact-us Below is my HTACCESS code in public folder which i got from internet. Mod rewrite is also loaded, i checked the PhPInfo().

<IfModule mod_rewrite.c>
    Options +FollowSymLinks -Indexes
    RewriteEngine On

    RewriteBase /

    # Handle Authorization Header
    RewriteCond %{HTTP:Authorization} .
    RewriteRule .* - [E=HTTP_AUTHORIZATION:%{HTTP:Authorization}]

    RewriteCond %{HTTP_HOST} !^www\. [NC]
    RewriteRule ^(.*)$ https://www.mywebsite.in/ [L,R=301]

    RewriteCond %{HTTPS} off    
    RewriteRule ^(.*)$ https://www.mywebsite.in/ [L,R=301]

    # Redirect Trailing Slashes If Not A Folder...
    RewriteCond %{REQUEST_FILENAME} !-d
    RewriteCond %{REQUEST_URI} (.+)/$
    RewriteRule ^ %1 [L,R=301]

    # Handle Front Controller...
    RewriteCond %{REQUEST_FILENAME} !-d
    RewriteCond %{REQUEST_FILENAME} !-f
    RewriteRule ^ index.php [L] 
    #RewriteCond %{THE_REQUEST} ^GET.*index\.php [NC]
    #RewriteRule (.*?)index\.php/*(.*) / [R=301,NE,L]
</IfModule>

If i uncomment the last 2 lines, the issue gets resolved but another one arises.

We have a custom built CMS developed in Codeigniter. This entire CMS folder is kept inside the public folder so it can be accessed as https://www.mywebsite.in/cms/. If I UNCOMMENT the last 2 lines in IfModule, any url in the CMS is not working. Laravel gives 404 page not found error. If I COMMENT them, the CMS works fine but website url gets accessible by index.php too.

Please note there is no HTACCESS file inside CMS folder.

I've tried searching the web but couldn't find any solution to this. Also, I'm having nearly zero knowledge about HTACCESS rules

Can any one please help me out as soon as possible 'coz the website is LIVE ???

Much Regards and Thanks to All,

Javed