30756 Commits

Author SHA1 Message Date
Rohit Yadav
00e1b462f6 CLOUDSTACK-9591: Fix guest VM ovf xml to remove network nodes
This removes network details from the guest VM template OVF xml before deploying
a VM which would fail in case of dvswitch-based vmware environment with no
dummy/existing vswitch.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2017-03-30 18:01:46 +05:30
Rajani Karuturi
0f0d908292 Merge pull request #2019 from Accelerite/cs-9850
CLOUDSTACK-9851 travis CI build failure after merge of PR#1953 & CLOUDSTACK-9850Fixed travis CI failures happening after merge of PR#1953.
Also fixed root volume detach that was broken after merge of PR#1953.
I found following logs with root volume attach.

`
2017-03-24 23:19:48,380 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-27:ctx-7c53d1bc job-44) (logid:f3b54257) Executing AsyncJobVO {id:44, userId: 2, accountId: 2, instanceType: Volume, instanceId: 3, cmd: org.apache.cloudstack.api.command.admin.volume.AttachVolumeCmdByAdmin, cmdInfo: {"apiKey":"PYHMg5HhOkmTQvlmZSe1dIDfTxl8uwreeFoqfj0a4zPCC-1XTuwH7H5I8Ac9fyvPbtNO1eN9THi72q8ATfs8hg","signature":"DD+ljbVWpWQw6FCl1tCDSYe5Ui0\u003d","httpmethod":"GET","deviceid":"0","ctxAccountId":"2","uuid":"2db57cd8-c96c-48f9-83c5-b9a79c852cc5","cmdEventType":"VOLUME.ATTACH","virtualmachineid":"9cb6a944-8775-4e11-961b-0e8dbdebf5a7","response":"json","ctxUserId":"2","ctxStartEventId":"117","id":"2db57cd8-c96c-48f9-83c5-b9a79c852cc5","ctxDetails":"{\"interface com.cloud.vm.VirtualMachine\":\"9cb6a944-8775-4e11-961b-0e8dbdebf5a7\",\"interface com.cloud.storage.Volume\":\"2db57cd8-c96c-48f9-83c5-b9a79c852cc5\"}"}, cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0, result: null, initMsid: 4278190080, completeMsid: null, lastUpdated: null, lastPolled: null, created: null }
2017-03-24 23:19:50,160 ERROR [c.c.v.VmWorkJobHandlerProxy] (Work-Job-Executor-15:ctx-489eb1dd job-44/job-45 ctx-cce17070) (logid:f3b54257) Invocation exception, caused by: java.lang.RuntimeException: deviceId should be 1,2,4-5
`

* pr/2019:
  CLOUDSTACK-9851 travis CI build failure after merge of PR#1953

Signed-off-by: Rajani Karuturi <rajani.karuturi@accelerite.com>
2017-03-30 13:48:48 +05:30
Rohit Yadav
92fd5bee3d CLOUDSTACK-9591: Fix systemvmtemplate to not include network details
This removes nic/network specific details while exporting the systemvmtemplate
for vmware (ova file). Having this causes the ssvms to not deploy in
dvswitch-based vmware environments that have no vswitch portgroups (dummy etc).
Tested this on a local Trillian env.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2017-03-30 13:47:53 +05:30
Sudharma Jain
7348f9412a CLOUDSTACK-9851 travis CI build failure after merge of PR#1953 2017-03-30 09:22:58 +05:30
Rajani Karuturi
525c45c1e5 Merge pull request #1994 from nvazquez/CLOUDSTACK-9827
CLOUDSTACK-9827: Storage tags stored in multiple placesIssue description: https://issues.apache.org/jira/browse/CLOUDSTACK-9827

### Fixes
- Create Primary Storage: Persist tags into `storage_pool_tags` instead of `storage_pool_details`
- List Storage Tags: Queries `storage_pool_tags` table instead of `storage_tag_view`
- Find Storage Pools by Tags using proper DAO
- Remove storage tags after deleting Primary Storage
- Remove unused `StorageTagDao`, `StorageTagDaoImpl`, `StorageTagVO` and `storage_tag_view`

* pr/1994:
  CLOUDSTACK-9827: Storage tags stored in multiple places

Signed-off-by: Rajani Karuturi <rajani.karuturi@accelerite.com>
2017-03-28 11:09:56 +05:30
Rajani Karuturi
45dda46fed Merge pull request #1961 from nvazquez/addNewNfsTestData
Fix for test_snapshots.py using nfs2 instead of nfs templateFix for marvin test failure introduced in #1847

Cc: @borisstoyanov @rhtyd @karuturi

* pr/1961:
  Fix for test failure
  Fix for test_snapshots.py using nfs2 instead of nfs template

Signed-off-by: Rajani Karuturi <rajani.karuturi@accelerite.com>
2017-03-28 11:09:30 +05:30
Bharat Kumar
422787e2d9 added some logging and made an improvement to get_master_and_backupRouter method 2017-03-27 05:53:58 -07:00
Bharat Kumar
55067a8692 CLOUDSTACK-9726 Update state is not changed to UPDATE_FAILED in case when Host is put in Maintenance Mode. 2017-03-27 05:44:05 -07:00
Rohit Yadav
0ddcf0b175 CLOUDSTACK-9805: Display VR list in network details
Displays a VR tab that lists VRs for the network in the detail views for
isolated networks, shared networks and for VPCs.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2017-03-26 14:27:22 +05:30
nvazquez
edf0e2b26f CLOUDSTACK-9827: Storage tags stored in multiple places 2017-03-24 13:37:04 -03:00
cloudsadhu
9cc66a3609 BUG-ID:CLOUDSTACK-9829:update the smoketest for rootvolume to implement full clone logic 2017-03-23 17:42:44 +05:30
Rajani Karuturi
ec2d4dd422 Merge release branch 4.9 to master
* 4.9:
  CLOUDSTACK-9811: fix duplicated nics on VR caused by nic name p<slot_number>p<port_number>
2017-03-23 15:19:31 +05:30
Rajani Karuturi
9bf428162d Merge pull request #2011 from ustcweizhou/fix-issue-p55p1
CLOUDSTACK-9811: fix duplicated nics on VR caused by nic name p<slot_number>p<port_number>

* pr/2011:
  CLOUDSTACK-9811: fix duplicated nics on VR caused by nic name p<slot_number>p<port_number>

Signed-off-by: Rajani Karuturi <rajani.karuturi@accelerite.com>
2017-03-23 15:18:54 +05:30
Rajani Karuturi
2b0a5ae4b2 Merge pull request #2003 from swill/cs_ip_fix
CLOUDSTACK-9811: fixed an issue if the dev is not in the databagDefend against the specified dev not being in the databag.

* pr/2003:
  changed the order fix to be closer to the original code
  CLOUDSTACK-9811: fixed an issue if the dev is not in the databag

Signed-off-by: Rajani Karuturi <rajani.karuturi@accelerite.com>
2017-03-23 15:16:50 +05:30
Wei Zhou
bf93b6313e CLOUDSTACK-9811: fix duplicated nics on VR caused by nic name p<slot_number>p<port_number> 2017-03-20 07:32:31 +01:00
Will Stevens
bb4087733b changed the order fix to be closer to the original code 2017-03-17 15:07:11 -04:00
Anshul Gangwar
c68931fc64 CLOUDSTACK-9706: Added snapshots cleanup in start and storage GC thread if they are failed to cleanup during DeleteSnapshot command 2017-03-17 17:40:55 +05:30
Harikrishna Patnala
f34469a41b CLOUDSTACK-9112: deployVM thread is holding the global lock on network longer and cause delays and some improvements in the planner
There are some VM deployment failures happening when multiple VMs are deployed at a time, failures mainly due to NetworkModel code that iterates over all the vlans in the pod. This causes each deployVM thread to hold the global lock on Network longer and cause delays. This delay in turn causes more threads to choose same host and fail since capacity is not available on that host.

Following are some changes required to be done to reduce delays during VM deployments which in turn causes some vm deployment failures when multiple VMs are launched at a time.

    In Planner, remove the clusters that do not contain a host with matching service offering tag. This will save some iterations over clusters that dont have matching tagged host
    In NetworkModel, do not query the vlans for the pod within the loop. Also optimized the logic to query the ip/ipv6
    In DeploymentPlanningManagerImpl, do not process the affinity group if the plan has hostId provided.
2017-03-15 15:21:29 +05:30
Rajani Karuturi
1decf5366d Merge release branch 4.9 to master
* 4.9:
  CLOUDSTACK-9765: centos: packaging: fix missing path in spec file post install
2017-03-15 10:52:14 +05:30
Rajani Karuturi
929c8231c5 Merge pull request #1923 from resmo/fix/centos-spec-upgrade-db-properties
CLOUDSTACK-9765: centos: packaging: fix missing path in spec fileRelated to 6051fa503a2da7b96256f21e6b7c42aa1f3cb01e

/cc @rhtyd (shapeblue)

* pr/1923:
  CLOUDSTACK-9765: centos: packaging: fix missing path in spec file post install

Signed-off-by: Rajani Karuturi <rajani.karuturi@accelerite.com>
2017-03-15 10:51:36 +05:30
Will Stevens
8d4855b4eb CLOUDSTACK-9811: fixed an issue if the dev is not in the databag 2017-03-14 09:23:47 -04:00
Suresh Sadhu
871f21af35 CLOUDSTACK-9604: Root-volume resize additional tests. 2017-03-14 15:39:49 +05:30
Rajani Karuturi
ad7ed7a178 Merge pull request #847 from kishankavala/CLOUDSTACK-8880
Bug-ID: CLOUDSTACK-8880: calculate free memory on host before deploying Vm.  free memory = total memory - (all vm memory)With memory over-provisioning set to 1, when mgmt server starts VMs in parallel on one host, then the memory allocated on that kvm can be larger than the actual physcial memory of the kvm host.

Fixed by checking free memory on host before starting Vm.
Added test case to check memory usage on Host.
Verified Vm deploy on Host with enough capacity and also without capacity

* pr/847:
  Bug-ID: CLOUDSTACK-8880: calculate free memory on host before deploying Vm.  free memory = total memory - (all vm memory)

Signed-off-by: Rajani Karuturi <rajani.karuturi@accelerite.com>
2017-03-13 22:20:58 +05:30
Rajani Karuturi
3f0fbf251c Merge pull request #1953 from Accelerite/CLOUDSTACK-9794
CLOUDSTACK-9794: Unable to attach more than 14 devices to a VMUpdated hardcoded value with max data volumes limit from hypervisor capabilities.

* pr/1953:
  CLOUDSTACK-9794: Unable to attach more than 14 devices to a VM

Signed-off-by: Rajani Karuturi <rajani.karuturi@accelerite.com>
2017-03-13 22:19:04 +05:30
Rajani Karuturi
9b85cbca41 Merge pull request #1958 from shapeblue/CLOUDSTACK-5806
CLOUDSTACK-5806: add presetup to storage types that support over provisioning

Ideally this should be configurable via global settings

* pr/1958:
  CLOUDSTACK-5806: add presetup to storage types that support over provisioning Ideally this should be configurable via global settings

Signed-off-by: Rajani Karuturi <rajani.karuturi@accelerite.com>
2017-03-13 22:16:16 +05:30
Rajani Karuturi
09802e0f3c Merge pull request #1861 from sateesh-chodapuneedi/pr-cloudstack-9698
CLOUDSTACK-9698 [VMware] Make hardcorded wait timeout for NIC adapter hotplug as configurableJira
===
CLOUDSTACK-9698 [VMware] Make hardcoded wait timeout for NIC adapter hotplug as configurable

Description
=========
Currently ACS waits for 15 seconds (hard coded) for hot-plugged NIC in VR running on VMware to get detected by guest OS.
The time taken to detect hot plugged NIC in guest OS depends on type of VMware NIC adapter like (E1000, VMXNET3, E1000e etc.)
and guest OS itself. In uncommon scenarios the NIC detection may take longer time than 15 seconds,
in such cases NIC hotplug would be treated as failure which results in VPC tier configuration failure.
Alternatively making the wait timeout for NIC adapter hotplug as configurable will be helpful for admins in such scenarios. This is specific to VR running over VMware hypervisor.

Also in future if VMware introduces new NIC adapter types which may take time to get detected by guest OS, it is good to have flexibility of
configuring the wait timeout as fallback mechanism in such scenarios.

Fix
===
Introduce new configuration parameter (via ConfigKey) "vmware.nic.hotplug.wait.timeout" which is "Wait timeout (milli seconds) for hot plugged NIC of VM to be detected by guest OS." as fallback instead of hard coded timeout, to ensure flexibility for admins given the listed scenarios above.

Signed-off-by: Sateesh Chodapuneedi <sateesh.chodapuneedi@accelerite.com>

* pr/1861:
  CLOUDSTACK-9698 Make the wait timeout for NIC adapter hotplug as configurable

Signed-off-by: Rajani Karuturi <rajani.karuturi@accelerite.com>
2017-03-13 22:12:54 +05:30
Rajani Karuturi
56e851ca46 Merge release branch 4.9 to master
* 4.9:
  moved logrotate from cron.daily to cron.hourly for vpcrouter in cloud-early-config
  CLOUDSTACK-9569: propagate global configuration router.aggregation.command.each.timeout to KVM agent
2017-03-13 22:09:27 +05:30
Rajani Karuturi
7b719c71fc Merge pull request #1856 from ustcweizhou/set-kvm-host-params
[4.9] CLOUDSTACK-9569: propagate global configuration router.aggregation.command.each.timeout to KVM agentThe router.aggregation.command.each.timeout in global configuration is only applied on new created KVM host.
For existing KVM host, changing the value will not be effective.
We need to propagate the configuration to existing host when cloudstack-agent is connected.

* pr/1856:
  CLOUDSTACK-9569: propagate global configuration router.aggregation.command.each.timeout to KVM agent

Signed-off-by: Rajani Karuturi <rajani.karuturi@accelerite.com>
2017-03-13 22:08:47 +05:30
Suresh Kumar Anaparti
93f5b6e8a3 CLOUDSTACK-9794: Unable to attach more than 14 devices to a VM
Updated hardcoded value with max data volumes limit from hypervisor capabilities.
2017-03-13 16:14:12 +05:30
Suresh Kumar Anaparti
8676b20276 CLOUDSTACK-9720: [VMware] template_spool_ref table is not getting updated with correct template physical size in template_size column. 2017-03-13 00:48:49 +05:30
Nathan Johnson
5c476492b1 CLOUDSTACK-8239 - Adding support for virtio-scsi on KVM hosts
This adds support for virtio-scsi on KVM hosts, either
for guests that are associated with a new os_type of 'Other PV Virtio-SCSI (64-bit)',
or when a VM or template is regstered with a detail parameter rootDiskController=scsi.

Update cloudstack add template dialog to allow for selecting rootDiskController with KVM

Update cloudstack kvm virtio-scsi to enable discard=unmap
2017-03-12 10:54:43 -05:00
Anshul And Priyank
ec66256149 CLOUDSTACK-9604: Root disk resize support for VMware and XenServer. 2017-03-09 19:20:38 +05:30
Jayapal
e1384c3b63 CLOUDSTACK-8931: Fail to deploy VM instance when use.system.public.ips=false 2017-03-09 17:40:34 +05:30
Jayapal
d7eae25322 CLOUDSTACK-9208: Assertion Error in VM_POWER_STATE handler- Fixed 2017-03-09 16:00:16 +05:30
Abhinandan Prateek
e303eee8e6 CLOUDSTACK-9828: GetDomRVersionCommand fails to get the correct version as output
Fix tries to return the output as a single command, instead of appending output from two commands
2017-03-09 14:15:45 +05:30
Rajani Karuturi
850c07cc8a Merge pull request #1991 from Accelerite/CLOUDSTACK-9821
CLOUDSTACK-9821: Fixed issue in deploying vm in basic zoneFixed issue in deploying vm in basic zone.
There is issue in ipset command with xenserver 6.5. In util.pread2 ipset and -N is passed as single string and it caused the issue in command failure.

 util.pread2(['/bin/bash', '-c', 'ipset', '-N ',  tmpname , type])

* pr/1991:
  CLOUDSTACK-9821: Fixed issue in deploying vm in basic zone

Signed-off-by: Rajani Karuturi <rajani.karuturi@accelerite.com>
2017-03-08 16:16:21 +05:30
Rajani Karuturi
40e49a626f Merge pull request #1982 from Accelerite/CLOUDSTACK-9807
CLOUDSTACK-9807/CLOUDSTACK-9808 - Added upgrade changes for 4.10 system vm templateRemoved systemvm template upgrade code in 4.6 upgrade. Added 4.10 system vm template upgrade changes with new urls and md5sum

Manual testing results:
Updated min req version:
mysql> select name, value from configuration where name = 'minreq.sysvmtemplate.version';
+------------------------------+----------+
| name                         | value    |
+------------------------------+----------+
| minreq.sysvmtemplate.version | 4.10.0.0 |
+------------------------------+----------+
1 row in set (0.00 sec)

Update template for KVM:
mysql> select * from vm_template where name = 'systemvm-kvm-4.10'\G
*************************** 1. row ***************************
                  id: 201
         unique_name: 201-2-badc363a-507d-3802-942f-205d35c3b575
                name: systemvm-kvm-4.10
                uuid: 3526258d-d9a4-4c3d-8f15-3137eb5d7ab8
              public: 0
            featured: 0
                type: SYSTEM
                 hvm: 1
                bits: 64
                 url: http://packages.shapeblue.com/systemvmtemplate/4.10/systemvm64template-4.10-kvm.qcow2.bz2
              format: QCOW2
             created: 2017-03-02 11:35:06
             removed: NULL
          account_id: 2
            checksum: NULL
        display_text: systemvm-kvm-4.10
     enable_password: 0
       enable_sshkey: 0
         guest_os_id: 184
            bootable: 1
         prepopulate: 0
         cross_zones: 1
         extractable: 0
     hypervisor_type: KVM
  source_template_id: NULL
        template_tag: NULL
            sort_key: 0
                size: 5242880
               state: Active
        update_count: 0
             updated: NULL
dynamically_scalable: 0
1 row in set (0.00 sec)

* pr/1982:
  Bug-ID: CLOUDSTACK-9807: 4.10.0.0 systemvm template upgrade.

Signed-off-by: Rajani Karuturi <rajani.karuturi@accelerite.com>
2017-03-08 16:11:29 +05:30
Nitin Kumar Maharana
63f534f292 CLOUDSTACK-9611: Dedicating a Guest VLAN range to Project does not work. 2017-03-08 02:06:41 +05:30
Pranali Mande
616c05e292 CLOUDSTACK-9824:Resource count for Primary storage is considered twice - while creating and while attaching the disk 2017-03-07 17:23:02 +05:30
Kishan Kavala
1ed7e3d6b0 Bug-ID: CLOUDSTACK-9807: 4.10.0.0 systemvm template upgrade.
Removed systemvm template upgrade code on 4.6 upgrade. Added 4.10 system
vm template upgrade changes with new urls and md5sum
2017-03-07 15:26:55 +05:30
Priyank Parihar
4b165f1b8f CLOUDSTACK-9610: Disabled Host Keeps Being up status after unmanging cluster. 2017-03-07 14:23:29 +05:30
Jayapal
e3ae08b3ee CLOUDSTACK-9709: Updated the vm ip fetch task to use the correct the thread 2017-03-07 09:50:18 +05:30
Jayapal
43a991d2eb CLOUDSTACK-9821: Fixed issue in deploying vm in basic zone 2017-03-06 19:01:15 +05:30
Nitesh Sarda
254771c01c CLOUDSTACK-9814 : Unable to edit a Sub domain, which has the same name in different domains 2017-03-03 20:01:19 +05:30
SudharmaJain
9d268c8cd5 CLOUDSTACK-8910: The reserved_capacity field increases suddenly after a vmware host failure
In case of vmware host failure, all the VMs including stopped VMs migrate
to the new host. For the Stopped Vms powerhost gets updated. This was
triggering HandlePowerStateReport which finally calls updatePowerState
updating update_time for the VM. This cause the capacity being reserved
for stopped VMs.
2017-03-03 18:56:58 +05:30
David Mabry
3e54388858 moved logrotate from cron.daily to cron.hourly for vpcrouter in cloud-early-config
(cherry picked from commit 450deed8472c06c4fec88e045bef28091502c638)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2017-03-03 14:00:43 +05:30
nvazquez
c66df6e11f Fix for test failure 2017-03-02 16:14:45 -03:00
Jayapal
d71879c3a7 CLOUDSTACK-9756: Configure to ignore the ipassoc failure 2017-03-02 14:33:33 +05:30
Sateesh Chodapuneedi
d171bb7857 CLOUDSTACK-9698 Make the wait timeout for NIC adapter hotplug as configurable
Currently ACS waits for 15 seconds (hard coded) for hot-plugged NIC in VR to get detected by guest OS.
The time taken to detect hot plugged NIC in guest OS depends on type of NIC adapter like (E1000, VMXNET3, E1000e etc.)
and guest OS itself. In uncommon scenarios the NIC detection may take longer time than 15 seconds,
in such cases NIC hotplug would be treated as failure which results in VPC tier configuration failure.
Alternatively making the wait timeout for NIC adapter hotplug as configurable will be helpful for admins in such scenarios.

Also in future if VMware introduces new NIC adapter types which may take time to get detected by guest OS, it is good to have flexibility of
configuring the wait timeout as fallback mechanism in such scenarios.

Signed-off-by: Sateesh Chodapuneedi <sateesh.chodapuneedi@accelerite.com>
2017-03-02 03:04:53 +05:30
Sateesh Chodapuneedi
9bed11d1f1 CLOUDSTACK-3223 Exception observed while creating CPVM in VMware Setup with DVS
Issue
=====
Exception observed while creating CPVM in VMware Setup with DVS.
Observed error "StartCommand failed due to Exception: com.vmware.vim25.AlreadyExists."
This is due to concurrent attempts to create same dv portgroup on same dvSwitch by
manager threads of CPVM and SSVM when both are started at same time.

Fix
===
Synchronize api calls to create/update dvportgroup.
Also maintaing local cache to avoid multiple fetch api calls to vCenter
when multiple threads try to create same object.

Signed-off-by: Sateesh Chodapuneedi <sateesh.chodapuneedi@accelerite.com>
2017-03-02 02:48:06 +05:30