31109 Commits

Author SHA1 Message Date
Rafael Weingärtner
3c6df7c970 CLOUDSTACK-10196: Remove ejb-api 3.0 dependency (#2348)
Fixed QuotaAlertManagerImplTest, which was injecting mock objects manually.
2017-12-18 23:59:06 +05:30
ming416
973a9c1dce CLOUDSTACK-10003 automatic configure juniper srx/vsrx nat loopback, (#2184)
Automatic configure juniper srx/vsrx nat loopback,

Constraint condition that manual configure source nat in juniper srx,allowed vm vist public network :
zone : trust to {trust,untrust}
rule : source address {0.0.0.0/0},destination address{0.0.0.0/0} ,do source nat with pool {public network getway ip}.

code change for trust to trust destination or static nat:

1. add srxCommand :CHECK_PRIVATE_IF_EXISTS. for add/delete rule to detect whether exist or not contain DestinationNatRule or StaticNatRule (ruleName_private) in trust zone .
2. add DestinationNatRule  (ruleName_private) to trust zone when ADD DestinationNatRule to untrust .
3. delete DestinationNatRule  (ruleName_private) from trust zone when DELETE DestinationNatRule from untrust.
4. add StaticNatRule (ruleName_private) to trust zone when ADD  StaticNatRule to untrust .
5. delete  StaticNatRule (ruleName_private) from trust zone when DELETE  StaticNatRule from untrust.

Check private if exist.
2017-12-16 09:51:25 +05:30
subhash yedugundla
566238b08e CLOUDSTACK-8908: After copying the template charging for that template is stopped (#2308)
When a template is copied back to zone after it is deleted. deleted field gets reset to null. delete field is added to Search on template zone mapping table to take care of the existing mapping.
2017-12-16 09:50:11 +05:30
Syed Mushtaq Ahmed
47419df368 CLOUDSTACK-10039: Adding used IOPS to storage pool response (#2294)
This change adds allocatediops to the ListStoragePool API. This applies to managed storage where we have a guaranteed minimum IOPS set. This is useful for monitoring if we have reached the IOPS limit on a storage cluster.
2017-12-15 18:12:13 +05:30
Paul Angus
b417226d05 CLOUDSTACK-9839: Add XenServer 7.1 and 7.2 interoperablility (#2346)
Adds XenServer 7.1 and 7.2 interoperablility.
Adds hypervisor to capabilities list and add guest OS mappings (copied from 7.0.0 mappings initially).
Add newly supported OSes and remove deprecated ones (XenServer)
2017-12-15 17:37:19 +05:30
niteshsarda
44a7eb0d14 CLOUDSTACK-10176: VM Start Api Job returns success for failed Job (#2354)
ISSUE :

VM start API Job returns success for failed Job

STEPS TO REPRODUCE :

Stop VM instance.
Mark state of associate router as Stopping from DB
Execute startVirtualMachine Api from rest client.
VM will not start but still main job will return the status as SUCCEEDED, whereas sub-job for command VmWorkStart will return status as failed.
2017-12-15 17:18:51 +05:30
Rohit Yadav
fc49533a8e
CLOUDSTACK-9986: Use host table to iterate for zone/cluster metrics (#2353)
The host_view may contain duplicate entries when hosts have tags.
Changing the host_view may cause unseen regressions so
to fix the issues we've modified the zone/cluster metrics code to use
the `host` table (hostdao) to iterate through the list of hosts in a
cluster during zone/cluster metrics listing.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2017-12-15 17:17:17 +05:30
Marc-Aurèle Brothier
d57b518727
Merge pull request #2359 from exoscale/doc_virutal
doc: replace virutal by virtual (typo)
2017-12-14 06:28:21 +01:00
Yoan Blanc
6ab2ba9230
doc: replace virutal by virtual (typo)
Signed-off-by: Yoan Blanc <yoan.blanc@exoscale.ch>
2017-12-13 09:59:15 +01:00
Wido den Hollander
885f561791 CLOUDSTACK-8313: Support overprovisioning for local storage (#2356)
This used to work in earlier version but was broken during a earlier
refactor.

This commit restores the functionality, but is also adds a boolean
per storage pool type if it supports overprovisioning.

This removes the nasty if which hard codes it.

UnitTests are also added to this commit to verify this functionality
remains stable in future releases.

Signed-off-by: Wido den Hollander <wido@widodh.nl>
2017-12-12 20:57:21 +05:30
Milamber
895aee37f7 CLOUDSTACK-9736: Incoherent validation and error message when you change the vm.password.length configuration parameter (#2358)
Default value introduce in schema-430to440.sql are 6 for the length

CLOUDSTACK-10111 have already the error message.
2017-12-12 17:06:20 +05:30
Rafael Weingärtner
bac423fea0 CLOUDSTACK-10184: Re-work method QuotaResponseBuilderImpl.startOfNextDay (#2326)
Also, removed @Local annotation that is not needed and add test cases.
2017-12-11 22:32:23 +05:30
René Moser
e5f0a68a4d CLOUDSTACK-10077: allow to have different VPN customer gateway configs for same gateway IP (#2267)
Related 86bbe211f2 and CLOUDSTACK-494. Currently we can not have serveral VPCs in one account with different VPN customer gateways configuration per same gateway IP.
2017-12-11 21:58:21 +05:30
Nitin Kumar Maharana
8c6fb0e312 CLOUDSTACK-10157: Wrong notification while migration (#2337)
Root Cause:
Earlier, it was failing with ArrayIndexOutOfBoundsException, when the list is empty and accessing the first element.
The error was only observed in Log, but was not showing in UI as it was not throwing any exception.
Hence the API call was in turn successful.

Solution:
Added the empty check before sending device details.
Which says either the required GPU device is not available or out of capacity.
2017-12-11 21:50:37 +05:30
Wido den Hollander
be3a39ea4f CLOUDSTACK-10177: Only pass IPv6 address to Security Group Python script if present (#2355)
Otherwise we send down a 'null' to a ProcessBuilder in Java instead of a String and this
causes a NPE.

We should check first if the Instance has a IPv6 address before sending it there.

Signed-off-by: Wido den Hollander <wido@widodh.nl>
2017-12-11 21:47:50 +05:30
Rohit Yadav
637d9a62c5
CLOUDSTACK-10070: Rearrange tests to avoid timeout (#2349)
This rearranges tests in travis.yml to avoid Travis test job timeout.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2017-12-08 21:52:22 +05:30
subhash yedugundla
ed6e3be756 CLOUDSTACK-7793: Create Snaphot with quiesce option set to true fails with InvalidParameterValueException (#2312)
reate Snaphot with quiesce option set to true fails with InvalidParameterValueException

--quiescevm option is only supported with netapp plugin with vmware. When netapp is not there, they would be struck in Allocated state and Deletion is not supported in allocated state. To fix this issue, deletion is supported in allocated state as well.
2017-12-01 02:43:39 +05:30
Sigert Goeminne
8f35657ac2 CLOUDSTACK-10164: allow users to create a VPC through the UI. (#2345)
The listNuageVspDomainTemplat cmd was called by the ui when the create vpc dialog was opened. This command failed when no nuage vsp device was present. As a consequence the ui did not show the dialog. So currently it's not possible to create a vpc through the UI with a native CloudStack deployment. This bugfix, adds robustness to the ui that in case the cmd fails the dialog will appear nonetheless . Furthermore I also changed the listNuageVspDomainTemplateCmd to always return an empty result when there is no nuage zone.
2017-12-01 02:39:21 +05:30
Boris Stoyanov - a.k.a Bobby
ef4adb3672 CLOUDSTACK-10070: Fixing some component tests and adding them in travis (#2263)
- Fixing some component tests and adding them in travis
- Rearranges tests in .travis.yml file

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2017-11-30 16:49:06 +05:30
Boris Stoyanov - a.k.a Bobby
919d298630 CLOUDSTACK-9586: Adding local storage test (#2342)
Added local storage test. Result:
==== Marvin Init Successful ====Host id fc9fa94f-7c78-41d0-b38c-6f5bfdf22571, hypervisor XenServer, localstorage False
Host id b54ccd0f-cd3c-4367-b9b4-f7d370b71670, hypervisor XenServer, localstorage True
Template id b8d66364-d5a0-11e7-8e52-06ca2801075c is Ready True
Prepare Template result [{domain : u'ROOT', domainid : u'b8d2a14e-d5a0-11e7-8e52-06ca2801075c', ostypename : u'CentOS 5.6 (64-bit)', zoneid : u'daf11016-d38b-4dae-aad1-b46bfefbd5e6', displaytext : u'CentOS 5.6(64-bit) no GUI (XenServer)', ostypeid : u'b8f78667-d5a0-11e7-8e52-06ca2801075c', passwordenabled : False, id : u'b8d66364-d5a0-11e7-8e52-06ca2801075c', size : 21474836480, isready : True, format : u'VHD', templatetype : u'BUILTIN', bits : 0, zonename : u'trl-1082-x-cs411-bstoyanov', status : u'Download Complete', isdynamicallyscalable : True, tags : [], isfeatured : True, sshkeyenabled : False, isextractable : True, crossZones : True, account : u'system', name : u'CentOS 5.6(64-bit) no GUI (XenServer)', created : u'2017-11-30T07:57:42+0000', hypervisor : u'XenServer', ispublic : True, checksum : u'905cec879afd9c9d22ecc8036131a180', physicalsize : 1758786048}]
=== TestName: test_01_prepare_template_local_storage | Status : SUCCESS ===
2017-11-30 14:07:51 +05:30
Paul Angus
e2aee99623 Revert "add XenServer base 7.1 and 7.2 capabilities to database"
This reverts commit 1421a6e07a76fe3a91cd342c9f43266305f23ff2.
2017-11-29 16:36:31 +00:00
Paul Angus
1421a6e07a add XenServer base 7.1 and 7.2 capabilities to database 2017-11-29 16:25:56 +00:00
Gabriel Beims Bräscher
bd5604428b
Merge pull request #2124 from GabrielBrascher/CLOUDSTACK-9432
CLOUDSTACK-9432: cluster/host dedicated to a domain is owned by the root domain
2017-11-28 10:31:23 -02:00
Bitworks Software, Ltd
bec48c62b6 CLOUDSTACK-10140: Fix for when template is created from snapshot template.properties are corrupted (#2322)
Fix for when template is created from snapshot template.properties are corrupted
2017-11-28 10:00:36 +05:30
Boris Stoyanov - a.k.a Bobby
f506a99df5 CLOUDSTACK-10154: fixing some smoketests failures (#2335)
* CLOUDSTACK-10154: fixing some smoketests failures

* Adding wait_until pattern to test_volumes
2017-11-28 09:55:35 +05:30
Wido den Hollander
fdf2509060 CLOUDSTACK-10160: Fix typo in Libvirt XML definition for Virtio-SCSI (#2341)
* CLOUDSTACK-10160: Fix typo in Libvirt XML definition for Virtio-SCSI

The attribute for the XML element 'controller' should be 'model' and
not 'mode'.

Source: https://libvirt.org/formatdomain.html#elementsControllers

  A scsi controller has an optional attribute model, which is one of
  'auto', 'buslogic', 'ibmvscsi', 'lsilogic', 'lsisas1068', 'lsisas1078',
  'virtio-scsi' or 'vmpvscsi'.

In the current state a regular SCSI device is attached and not a Virtio-SCSI
device.

Signed-off-by: Wido den Hollander <wido@widodh.nl>

* CLOUDSTACK-10160: Add UnitTest for LibvirtVMDef.SCSIDef

To make sure the XML output string is correct

Signed-off-by: Wido den Hollander <wido@widodh.nl>
2017-11-28 09:42:15 +05:30
Bitworks Software, Ltd
d65f0447ae CLOUDSTACK-10138: Load br_netfilter in security_group management script (#2321)
Improved debian installation script to find either rb_netfilter module exists in the system and should be loaded. The implementation does check and fix during installation process (Ubuntu 14.04/16.04)
2017-11-27 13:12:37 +05:30
Jayapal
88327580b0 CLOUDSTACK-10152: Skip the destination cidr in ipset when 0.0.0.0/0 pa… (#2334)
Skip the destination cidr in ipset when 0.0.0.0/0 passed.
Fixed adding issue with egress dest cidr with 0.0.0.0/0.
2017-11-23 11:42:43 +05:30
Rafael Weingärtner
cb9c7adddf CLOUDSTACK-10156: Fix Coverity new problems CID(1349987, 1349986, 1347248) (#2332)
Fix Coverity new problems CID(1349987, 1349986, 1347248).
2017-11-23 11:40:53 +05:30
Wido den Hollander
632479d8f8 CLOUDSTACK-9853: Add support for Secondary IPv6 Addresses and Subnets (#2028)
This commit adds support for passing IPv6 Addresses and/or Subnets as
Secondary IPs.

This is groundwork for CLOUDSTACK-9853 where IPv6 Subnets have to be
allowed in the Security Groups of Instances to we can add DHCPv6
Prefix Delegation.

Use ; instead of : for separating addresses, otherwise it would cause
problems with IPv6 Addresses.

Signed-off-by: Wido den Hollander <wido@widodh.nl>
2017-11-22 17:30:33 +05:30
subhash yedugundla
421517f949 CLOUDSTACK-10133: Local storage overprovisioning for ext file system (#2310)
Local storage overprovisioning for ext file system
2017-11-22 17:23:34 +05:30
Boris Stoyanov - a.k.a Bobby
fd7eeb4848 CLOUDSTACK-9989: Extend smoketests suite (#2219)
Adding tests to .travis, some of them were already there so just moving the reference.
2017-11-22 17:06:22 +05:30
Sigert Goeminne
77864992fe CLOUDSTACK-9776: extra DHCP options support for Nuage VSP
Co-Authored-By: Frank Maximus <frank.maximus@nuagenetworks.net>
Co-Authored-By: Prashanth Manthena <prashanth.manthena@nuagenetworks.net>
Co-Authored-By: Raf Smeets <raf.smeets@nuagenetworks.net>

Bug: https://issues.apache.org/jira/browse/CLOUDSTACK-9776

Design-Doc: https://cwiki.apache.org/confluence/display/CLOUDSTACK/CloudStack+extra+DHCP+option+support
2017-11-21 11:44:39 +01:00
Marc-Aurèle Brothier
421f1072d5 CLOUDSTACK-10123: Entity should use GMT TZ for timestamp values (#2303)
Depending on the timezone you're running CS (before GMT timezones) you could experience that some jobs are marked as failed since the parent job got a null result despite its child job having successfully done the job. The child job got deleted by the CleanupTask ahead of time, due to a missing datetime conversion to GMT timezone.

Jobs are failing with this message: Job failed with un-handled exception

The fix intends to correct any datetime used in the code that should be using the GMT timezone instead of the local one since all DB datetime should be stored at GMT.
2017-11-18 09:14:57 +05:30
Marc-Aurèle Brothier
cd6288ecfb CLOUDSTACK-10012: Jetty 9.4 (#2329)
* Bump Jetty to 9.4

* Use new jetty gzip handler

* Redirect / to context

* Update wiremock but still not working

* Add session timeout configuration

* server.properties.in: Change default timeout to 30 (mins)

* cloudian: fix unit test failures

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

* client: use older 9.2.x jetty-maven-plugin that works

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

* Moving jetty mvn plugin version in properties

Signed-off-by: Marc-Aurèle Brothier <m@brothier.org>

* Set default session timeout to 30mins
2017-11-17 23:26:02 +05:30
Frank Maximus
3eafd0ce52
Merge pull request #1981 from nuagenetworks/feature/nuage_vpc_selectable_domain_template
CLOUDSTACK-9806: Nuage domain template selection per VPC
2017-11-17 11:31:48 +01:00
Rene Moser
55c059c098 CLOUDSTACK-10144: fix possible NPE in listVlanIpRanges 2017-11-17 10:04:43 +05:30
Rafael Weingärtner
7f6ae15972 Remove annotation not needed at cloud-engine-storage-image (#2324)
remove bogus depends-on declaration at spring-engine-storage-snapshot-core-context
2017-11-16 23:11:15 +05:30
Frank Maximus
14ab69e19a
Merge pull request #2005 from nuagenetworks/feature/nw_offering_for_vpc
CLOUDSTACK-9450: Network Offering for VPC based on DB flag
2017-11-16 17:33:07 +01:00
Rohit Yadav
a1008f9ae2
CLOUDSTACK-10129: Show account, network info in VR list view (#2327)
This shows the owner account and network of a VR in the VR list view,
and for VPCs shows the VPC name and redundant state of the VPC rVR.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2017-11-16 18:39:57 +05:30
Rohit Yadav
475aa03bb8 Merge branch '4.10'
Forwards merge 4.9->4.10->master.
2017-11-16 11:16:15 +05:30
Rohit Yadav
330f24117c Merge branch '4.9' into 4.10 2017-11-16 11:15:32 +05:30
Rohit Yadav
4fda0c039e
CLOUDSTACK-9998: Fix metrics generation failurein prometheus exporter (#2325)
This fixes metrics generation failures when metrics capacities are null.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2017-11-16 10:13:53 +05:30
Sigert Goeminne
880e475f40 CLOUDSTACK-9806: Nuage domain template selection per VPC
Co-Authored-By: Prashanth Manthena <prashanth.manthena@nuagenetworks.net>
Co-Authored-By: Frank Maximus <frank.maximus@nuagenetworks.net>

Bug: https://issues.apache.org/jira/browse/CLOUDSTACK-9806

Design Document: https://cwiki.apache.org/confluence/display/CLOUDSTACK/Nuage+domain+template+selection+per+VPC
2017-11-15 14:40:30 +01:00
subhash yedugundla
4f1539ee70 CLOUDSTACK-10135 ACL rules order is not maintained for ACL_OUTBOUND in VPC VR (#2313)
(cherry picked from commit 60bec39b9ddb72c9badda49dc39c4105596bcb90)
2017-11-15 10:12:18 +01:00
subhash yedugundla
f250b3ae0c CLOUDSTACK-10135 ACL rules order is not maintained for ACL_OUTBOUND in VPC VR (#2313)
(cherry picked from commit 60bec39b9ddb72c9badda49dc39c4105596bcb90)
2017-11-15 10:09:35 +01:00
subhash yedugundla
60bec39b9d CLOUDSTACK-10135 ACL rules order is not maintained for ACL_OUTBOUND in VPC VR (#2313) 2017-11-15 10:06:00 +01:00
Rohit Yadav
3ee8d83621 CLOUDSTACK-10136: Fix RemoteHostEndPoint thread growth
This fixes the following:
- Unchecked thread growth in RemoteEndHostEndPoint
- Potential NPE while finding EP for a storage/scope

Unbounded thread growth can be reproduced with following findings:
- Every unreachable template would produce 6 new threads (in a single
ScheduledExecutorService instance) spaced by 10 seconds
- Every reachable template url without the template would produce 1 new
thread (and one ScheduledExecutorService instance), it errors out quickly without
causing more thread growth.
- Every valid url will produce upto 10 threads as the same ep (endpoint
instance) will be reused to query upload/download (async callback)
progresses.

Every RemoteHostEndPoint instances creates its own
ScheduledExecutorService instance which is why in the jstack dump, we
see several threads that share the prefix RemoteHostEndPoint-{1..10}
(given poolsize is defined as 10, it uses suffixes 1-10).

This fixes the discovered thread leakage with following notes:
- Instead of ScheduledExecutorService instance, a cached pool could be
used instead and was implemented, and with `static` scope to be reused
among other future RemoteHostEndPoint instances.
- It was not clear why we would want to wait when we've Answers returned
from the remote EP, and therefore a scheduled/delayed Runnable was
not required at all for processing answers. ScheduledExecutorService
was therefore not really required, moved to ExecutorService instead.
- Another benefit of using a cached pool is that it will shutdown
threads if they are not used in 60 seconds, and they get re-used for
future runnable submissions.
- Caveat: the executor service is still unbounded, however, the use-case
that this method is used for short jobs to check upload/download
progresses fits the case here.
- Refactored CmdRunner to not use/reference objects from parent class.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2017-11-10 22:04:09 +05:30
Marc-Aurèle Brothier
9c067c0496 CLOUDSTACK-10137: Only link log4j if not already present (#2316)
Use ln -sf to add link log4j.xml.

Signed-off-by: Marc-Aurèle Brothier <m@brothier.org>
2017-11-10 19:59:59 +05:30
Jayapal
4af82bbc5c CLOUDSTACK-9961: Fixed adding domain in vpn customer gateway (#2157)
Added accepting domain for the s2s vpn customer vpn gateway.
2017-11-09 21:38:17 +05:30