Skip to main content

Cleaning or increasing the OnApp Integrated Storage DB size (onappstore cleandb)

  • Updated

onappstore cleandb is a utility for cleaning the /onappstore/DB partition or increasing its size. It was introduced in version 6.4.

On earlier versions, use the manual cleanup procedure below.

Recommendations

  • Keep the same DB size on all data store nodes: every Hypervisor, Backup Server and Storage Controller.
  • A Storage Controller with a 1 GB /onappstore/DB should have at least 4 GB of RAM.

Choose a method:

GoalMethod
Increase the DB size permanently on a Hypervisor and Storage Contoller, Backup Servers, reboot acceptableControl Panel
Clean or increase the DB on a Hypervisor or Backup Server, no rebootCommand line, frontend node
Clean or increase the DB on a Storage Controller, no rebootCommand line, Storage Controller (backend node)
onappstore getid or cleandb fails or hangsManual cleanup (Hypervisor, Storage Controller, Backup Server)

Increasing the DB size from the Control Panel (requires reboot)

  1. Go to Control Panel > Admin > Settings and click the Compute Resources or Backup Servers icon.
  2. Click Actions next to the compute resource or backup server and select Integrated Storage Settings.
  3. Click Edit.
  4. Change the Controller DB size field.
  5. Click Save.
  6. Reboot the compute resource.

If the required value is missing from the drop-down

To offer 512 / 1024 MB in the Controller DB size drop-down, add the values to /onapp/interface/config/info_hub.yml on the Control Panel server.

Before:

[root@cp178 ~]# grep -A5 db_size /onapp/interface/config/info_hub.yml
    db_size:
      default: 128
      values:
        - 128
        - 256
    disks_per_controller:

After:

[root@cp178 ~]# grep -A7 db_size /onapp/interface/config/info_hub.yml
    db_size:
      default: 128
      values:
        - 128
        - 256
        - 512   <===
        - 1024  <===
    disks_per_controller:

Then apply the change by clicking Save Configuration at http://<control-panel-ip>/settings/edit. If the new values still don’t appear, restart the OnApp services from the CLI.

Cleaning or increasing the DB size on the frontend node (no reboot)

Applies to Hypervisors and Backup Servers. No reboot is needed. On a Backup Server, repeat this procedure after every reboot, because the DB size reverts to the default and can’t be set from the UI.

  1. Verify that /onappstore/DB is full.
   [root@10.0.20.60 ~]# df -h | grep DB
   tmpfs  512M  512M  4.0K  100% /onappstore/DB
  1. Get the UUID of the frontend node (here 1463856283).
   [root@10.0.20.60 ~]# onappstore getid
   result=SUCCESS uuid=1463856283 ipaddr=10.0.20.60 backends=10.200.41.1 unicasthosts= completiontime=0
  1. Run the cleanup. To increase the size, set the new value in the size= parameter.
   [root@10.0.20.60 ~]# onappstore cleandb size=512 uuid=1463856283
   Clean DB at nodes dictkeys(['1463856283'])
   Stop ISD at nodes
   Successfully stopped ISD at all nodes
   Clean DB at nodes
   Successfully cleaned DB at all nodes
   Start ISD at nodes after cleanup
   Successfully started ISD at all nodes
   SUCCESS completion_time=0
  1. Verify the result.
   [root@10.0.20.60 ~]# df -h | grep DB
   tmpfs  512M  0  512M  0% /onappstore/DB

Cleaning or increasing the DB size on a Storage Controller (backend node)

  1. Get the backend IPs of the Storage Controller. Run onappstore getid on the frontend node. The backends= field lists the Storage Controller addresses (here 10.200.30.1 and 10.200.30.2).
   [root@192.168.70.101 ~]# onappstore getid
   result=SUCCESS uuid=1843932421 ipaddr=192.168.70.101 backends=10.200.30.1,10.200.30.2 unicasthosts= completion_time=7
  1. Connect to the Storage Controller and check the current DB size. Each /DB/NODE-<uuid> mount shows the UUID of a node on this controller.
   [root@192.168.70.101 ~]# telnet 10.200.30.2
   Trying 10.200.30.2...
   Connected to 10.200.30.2.
   Escape character is '^]'.

   / # df -h
   Filesystem                Size      Used Available Use% Mounted on
   /dev/vdisk2               3.5T      1.1T      2.3T  33% /DB/NODE-150070007
   /dev/vdisk1               3.5T    690.8G      2.8T  19% /DB/NODE-3150899262
   /dev/vdisk0               3.5T   1001.3G      2.5T  28% /DB/NODE-3496687535
   tmpfs                   512.0M    413.5M     98.5M  81% /onappstore/DB
  1. Run the cleanup with the UUID of any one node on this Storage Controller. Set the new value in size= (here 1024; make sure the controller has at least 4 GB of RAM).
   / # onappstore cleandb size=1024 uuid=150070007
   Clean DB at nodes dict_keys(['150070007'])
   Stop ISD at nodes
   Successfully stopped ISD at all nodes
   Clean DB at nodes
   Successfully cleaned DB at all nodes
   Start ISD at nodes after cleanup
   Successfully started ISD at all nodes
   SUCCESS completion_time=2
  1. Verify the result. 
   / # df -h
   Filesystem                Size      Used Available Use% Mounted on
   /dev/vdisk2               3.5T      1.1T      2.3T  33% /DB/NODE-150070007
   /dev/vdisk1               3.5T    690.8G      2.8T  19% /DB/NODE-3150899262
   /dev/vdisk0               3.5T   1001.3G      2.5T  28% /DB/NODE-3496687535
   tmpfs                     1.0G    992.0K   1023.0M   0% /onappstore/DB

Manual cleanup when onappstore commands fail

Use this procedure if:

  • onappstore getid fails and you cannot obtain the node UUID, or
  • onappstore cleandb fails or hangs.
  • You are using older version than 6.4

It applies to Hypervisors, Backup Servers and Storage Controllers. The steps are the same, only the shell differs.

  1. Stop the storage services and confirm they are no longer running.
   [root@10.0.20.60 ~]# /etc/init.d/storageAPI stop
   Stopping storageAPI (via systemctl): [ OK ]
   [root@10.0.20.60 ~]# /etc/init.d/isd stop
   Stopping ISD (PID 104221)…
  1. Remove the metadata.
   [root@10.0.20.60 ~]# rm -rf /onappstore/DB/v2/*
   [root@10.0.20.60 ~]# rm -rf /onappstore/DB/locks/*
  1. Start the storage services.
   [root@10.0.20.60 ~]# /etc/init.d/isd start
   Starting ISD…
   [root@10.0.20.60 ~]# /etc/init.d/storageAPI start
   Starting storageAPI (via systemctl): [ OK ]
  1. Verify that the DB is clean (df -h | grep DB) and that onappstore commands respond again. If you need a larger DB, repeat the relevant command-line procedure above with the new size= value.