2457 Commits

Author SHA1 Message Date
Abhishek Kumar
c671e07c18 server: update network offering for specified domain(s) & zone(s)
Added checkAccess while creating network from offering.
Response fixes for networkoffering APIs.

Signed-off-by: Abhishek Kumar <abhishek.kumar@shapeblue.com>
2019-05-24 10:11:00 +05:30
Abhishek Kumar
dc589a442d server: create network offering for specified domain(s) and zone(s)
Signed-off-by: Abhishek Kumar <abhishek.kumar@shapeblue.com>
2019-05-24 10:11:00 +05:30
Abhishek Kumar
977c5b7614 schema: domain and zone IDs in offering details hidden for user in upgrade
Signed-off-by: Abhishek Kumar <abhishek.kumar@shapeblue.com>
2019-05-24 10:10:59 +05:30
Abhishek Kumar
2102888116 server: changes for domain, zone specified service offerings
Added changes for creating service offerings for specified domain(s) and zone(s).
Fixed checkAccess for disk offerings.
Fixed list APIs for disk and service offerings.
UI changes for creating disk, service offerings for specified domain(s) and zone(s).

Signed-off-by: Abhishek Kumar <abhishek.kumar@shapeblue.com>
2019-05-24 10:09:52 +05:30
Abhishek Kumar
b8f1cf2648 schema: update service_offering details constraint
Signed-off-by: Abhishek Kumar <abhishek.kumar@shapeblue.com>
2019-05-24 10:04:10 +05:30
Rohit Yadav
94f425f6b1 refactored changes
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-05-24 10:04:10 +05:30
Rohit Yadav
f431711922 sql changes for service offering
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-05-24 10:04:10 +05:30
Rohit Yadav
d04c6ea276 remove getDomainId and write the workaround code
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-05-24 10:04:10 +05:30
Rohit Yadav
e3da9ec645 fix db issues
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-05-24 09:56:23 +05:30
Rohit Yadav
6a902460af find in set implementation
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-05-24 09:56:23 +05:30
Rohit Yadav
8cabbcea92 wip domainId removal stuff
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-05-24 09:56:23 +05:30
Rohit Yadav
5988884bd1 wip changes
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-05-24 09:56:23 +05:30
Rohit Yadav
7721126076 remove domain id usage
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-05-24 09:56:23 +05:30
Rohit Yadav
9c117f2781 DB upgrade path and UI fixes
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-05-24 09:56:23 +05:30
Rohit Yadav
728afba5d4 refactorings
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-05-24 09:56:23 +05:30
Abhishek Kumar
c85b3e597a server: ability to create disk offerings for domain(s) and zone(s)
Allows creating storage offerings associated with particular domain(s) and zone(s). In create disk/storage offfering form UI, a mult-select control has been addded to select desired zone(s) and domain select element has been made multi-select.
createDiskOffering API has been modified to allow passing list of domain and zone IDs with keys domainids and zoneids respectively. These lists are stored in DB in cloud.disk_offering_details table with 'domainids' and 'zoneids' key as string of comma separated list of IDs. Response for create, update and list disk offering APIs will return domainids, domainnames, zoneids and zonenames in details object of offering.
listDiskOfferings API has been modified to allow passing zoneid to return only offerings which are associated with the zone.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2019-05-24 09:56:23 +05:30
Daan Hoogland
8b75d61ffc Merge release branch 4.12 to master
* 4.12:
  KVM: Fix agents dont reconnect post maintenance (#3239)
2019-05-23 14:31:21 +02:00
Daan Hoogland
29918e25e3 Merge release branch 4.11 to 4.12
* 4.11:
  KVM: Fix agents dont reconnect post maintenance (#3239)
2019-05-23 14:29:41 +02:00
Nicolas Vazquez
e86f671c8e KVM: Fix agents dont reconnect post maintenance (#3239)
* Keep connection alive when on maintenance

* Refactor cancel maintenance and unit tests

* Add marvin tests

* Refactor

* Changing the way we get ssh credentials

* Add check on SSH restart and improve marvin tests
2019-05-23 14:13:17 +02:00
Abhishek Kumar
2020bfb6a3 server: allows compute offering with or without constraints (#3245)
Problem: Custom compute offering does not allow setting min and max values for CPU and VRAM for custom VMs.

Root Cause: Custom compute offerings cannot be created with a given range of CPU number and memory instead it allows only fixed values.

Solution: createServiceOffering API has been modified to allow setting a defined range for CPU number and memory. Also, UI form for compute offering creation is provided with a new field named 'compute offering type’ with values - Fixed, Custom Constrained, Custom Constrained. It will allow the creation of compute offerings either with a fixed CPU speed and memory for fixed compute offering, or with a range of CPU number and memory for custom constrained compute offering or without predefined CPU number, CPU speed and memory for custom unconstrained compute offering.

To allow the user to set CPU number, CPU speed and memory during VM deployment, UI form for VM deployment has been modified to provide controls to change these values. These controls are depicted in screenshots below for custom constrained and custom unconstrained compute offering types.

Sample API calls using cmk to create a constrained service offering and deploying a VM using it,

create serviceoffering name=Constrained displaytext=Constrained customized=true mincpunumber=2 maxcpunumber=4 cpuspeed=400 minmemory=256 maxmemory=1024

deploy virtualmachine displayname=ConstrainedVM serviceofferingid=60f3e500-6559-40b2-9a61-2192891c2bd6 templateid=8e0f4a3e-601b-11e9-9df4-a0afbd4a2d60 zoneid=9612a0c6-ed28-4fae-9a48-6eb207af29e3 details[0].cpuNumber=3 details[0].memory=800

Signed-off-by: Abhishek Kumar <abhishek.kumar@shapeblue.com>
2019-05-23 11:47:53 +05:30
Rohit Yadav
0700d91a68 Merge branch '4.12'
- Fixes PR #3146 db cleanup to the correct 4.12->4.13 upgrade path
- Fixes failing unit test due to jdk specific changes after forward
  merging

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-05-14 15:15:17 +05:30
Frank Maximus
e11f7ee1ba RIP Nuage Cloudstack Plugin (#3146)
may it rest in peaces
2019-05-14 10:58:24 +02:00
Rohit Yadav
00ff536f81 Merge remote-tracking branch 'origin/4.11' into 4.12
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-05-14 14:26:11 +05:30
skattoju4
4c60a5b1ff Fix slow vm creation when large sf snapshot count (#3282)
* skip geting used bytes for volumes that are not in Ready state
* updated log message
* filter snapshots by state backedup
* removed * import
* filter templates by state 'DOWNLOADED'
* refactored getUsedBytes to use O(1) queries
* querying for ready volumes instead filtering in memory
* make listByStoreIdInReadyState more generic ex listByStoreIdAndState
* updated snapshot search criteria for listByStoreIdAndState
* updated template search criteria for listByPoolIdAndState
* fixed typo in search criteria for listByTemplateAndState
* fixed typo in search criteria for templates in listByPoolIdAndState
2019-05-11 16:02:52 +02:00
Anurag Awasthi
f9b61bc737 orchestration: Allow VM that has never started to have volumes attached (#3276)
With this patch b766bf7
we started tracking disks in attaching state so that other attach request can fail gracefully. However this missed the case where disks were in allocated state but attach was requested.

For the use case where users want to attach disk in allocated state but not ready, we need to have allocated-attaching transition as well. We must take care of returning to the original state - allocated or ready - when attach request has completed.

For the use case of unstarted vm's the disk must proceed as follows - "Allocated" -> Attaching -> Allocated. When VM is started, the disk is "created" and pool is assigned. For the use case of started VMs it's more trivial and disk proceeds as follows - Ready -> Attaching -> Ready.

Test this by creating a VM with "startvm=false", create a disk and try attaching it in allocated state. It would give an exception on latest 4.11 but will be fixed on this patch.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-05-10 23:40:38 +05:30
ustcweizhou
b60daf7142 server: Fix exception while update domain resource count (#3204) 2019-04-29 08:51:08 +02:00
Rohit Yadav
491a10be0c
storage: publish delete usage event for snapshot deletion (#3212)
Problem: Users are billed for destroyed VMs with VM snapshots because usage records don't get that the VM and VM snapshots are removed.
Root Cause: The destroyVirtualMachine and expungeVirtualMachine APIs were removing VM snapshots but not generating VMSNAPSHOT.DELETE usage event due to which the VM snapshots were not marked as removed in the usage_vmsnapshot table.
Solution: The issue was fixed by emitting the proper usage event for all the VM snapshots of a VM that is destroyed.
2019-04-10 17:12:55 +05:30
Rohit Yadav
671a70a9a5
schema: add empty DB upgrade path from 4.12.0.0 to 4.13.0.0 (#3236)
This adds empty empty upgrade path from 4.12.0.0 to 4.13.0.0.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-03-27 00:00:47 +05:30
GabrielBrascher
8d3feb100a Updating pom.xml version numbers for release 4.13.0.0-SNAPSHOT
Signed-off-by: GabrielBrascher <gabriel@pcextreme.nl>
2019-03-20 18:47:35 -03:00
GabrielBrascher
a137398bf1 Updating pom.xml version numbers for release 4.12.0.0
Signed-off-by: GabrielBrascher <gabriel@pcextreme.nl>
2019-03-14 10:11:46 -03:00
Gabriel Beims Bräscher
7c5eca9481
Copy template to target KVM host if needed when migrating local <> local storage (#3154)
* Migrate template to target host if needed.

Fix KVM VM local storage live migration by migrating its template to the
target host if needed.

* Address reviewer and add method that updates the DB template reference

* Remove deprecated Config.PrimaryStorageDownloadWait

* Code formating of @Inject to follow checkstyle
2019-02-05 00:18:29 -02:00
Nathan Johnson
637cc6ec4e feature: add libvirt / qemu io bursting (#3133)
* feature: add libvirt / qemu io bursting

Adds the ability to set bursting features from libvirt / qemu

This allows you to utilize the iops and bytes temporary "burst" mode
introduced with libvirt 2.4 and improved upon with libvirt 2.6.

https://blogs.igalia.com/berto/2016/05/24/io-bursts-with-qemu-2-6/

* updates per rafael et al
2019-02-04 19:47:44 -02:00
GabrielBrascher
460d3127ec Fix conflict and merge forward PR #3122 from 4.11 to master (4.12) 2019-02-04 19:24:59 -02:00
Nathan Johnson
bf805d1483 Add back ability to disable backup of snapshot to secondary (#3122)
* The snapshot.backup.rightafter configuration variable was removed by:

SHA: 6bb0ca2f854

This adds it back, though named snapshot.backup.to.secondary now instead.

This global parameter, once set, will allow you to prevent automatic backups of
     snapshots to secondary storage, unless they're actually needed.

Fixes #3096

* updates per review
2019-02-04 19:08:42 -02:00
dahn
58466c8954 api: add command to list management servers (#3150)
* api: add command to list management servers

* api: add number of mangement servers in listInfrastructure command

* ui: add block for mangement servers on infra page

* api name resolution method cleanup
2019-02-01 16:23:39 -02:00
Wido den Hollander
9c4292cc45 network: Offerings do not have to have Security Grouping enabled (#3112)
Offerings can co-exist where on does provide Security Grouping in the
network, but other guest Networks have no Security Grouping.

In V(X)LAN isolation environments the L2 separation is handled by V(X)LAN
and protection between Instances is handled by Security Grouping.

There are multiple scenarios possible where one network has Security Grouping
enabled because that is required in that network.

In the other network, but in the same zone it could be a choice to have
Security Grouping disabled and allow all traffic to flow.

Signed-off-by: Wido den Hollander <wido@widodh.nl>
2019-01-28 15:38:08 -02:00
dahn
b363fd49f7 Vmware offline migration (#2848)
* - Offline VM and Volume migration on Vmware hypervisor hosts
- Also add VM disk consolidation call on successful VM migrations

* Fix indentation of marvin test file and reformat against PEP8

* * Fix few comment typos
* Refactor debug messages to use String.format() when debug log level is enabled.

* Send list of commands returned by hypervisor Guru instead of explicitly selecting the first one

* Fix unhandled NPE during VM migration

* Revert back to distinct event descriptions for VM to host or storage pool migration

* Reformat test_primary_storage file against PEP-8 and Remove unused imports

* Revert back the deprecation messages in the custom StringUtils class to favour the use of the ApacheUtils
2019-01-25 10:05:13 -02:00
Malcolm Taylor
97ddd8dffd Fix some LGTM alerts (#3143)
* Fix some LGTM alerts

* address review comments from Gabriel Brascher and Rafael Weingartner
2019-01-24 20:22:39 -02:00
Matheus Marabesi
2d37b746b8 db: alter cloud.sslcerts fingerprint column from varchar(62) to text (#3132)
As described in the issue #3123 the certificate endpoint throws an exception when a request is made to create the certificate.
Fixes #3123
2019-01-17 13:26:22 +05:30
Wido den Hollander
af9fefc6b4 ipv6: Calculate IPv6 address instead of fetching one from a pool (#3077)
With IPv6 we are not using DHCP to allocate addresses, but using
StateLess Address Auto Configuration (SLAAC) a Instance will calculate
it's own address based on the Router Advertisements send out by the
routers in the network.

This Advertisement contains the IPv6 Subnet in use in that subnet and
allows to calculate the stable Address the Instance will obtain based
on it's MAC Address.

The existing code is 'dead code' as it has been written, but was never
used by any production code.

SLAAC only works properly with subnets of exactly 64-bits large.

Signed-off-by: Wido den Hollander <wido@widodh.nl>
2019-01-16 16:49:02 -02:00
Rohit Yadav
8849382f77
db: Fixes #2935 GC MySQL error (#3115)
This fixes the variable name to adhere to the DB framework convention
as defined by the method interceptor that creates attributes:

https://github.com/apache/cloudstack/blob/master/framework/db/src/main/java/com/cloud/utils/db/SearchBase.java#L480

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-01-13 11:16:57 +05:30
Marc-Aurèle Brothier
cea8036732 VMTemplateZone needs some love (#1730)
It's incorrect to use the findIncludingRemovedBy and
listIncludingRemovedBy for the common list and find operation.

Signed-off-by: Marc-Aurèle Brothier <m@brothier.org>
2019-01-10 08:55:09 -02:00
Rohit Yadav
92cc4514ea Merge remote-tracking branch 'origin/4.11'
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-12-28 15:20:23 +05:30
Nicolas Vazquez
13c81a8ee4 server: Prevent corner case for infinite PrepareForMaintenance (#3095)
A corner case was found on 4.11.2 for #2493 leading to an infinite loop in state PrepareForMaintenance

To prevent such cases, in which failed migrations are detected but still running on the host, this feature adds a new cluster setting host.maintenance.retries which is the number of retries before marking the host as ErrorInMaintenance if migration errors persist.

How Has This Been Tested?
- 2 KVM hosts, pick one which has running VMs as H
- Block migrations ports on H to simulate failures on migrations:
iptables -I OUTPUT -j REJECT -m state --state NEW -m tcp -p tcp --dport 49152:49215 -m comment --comment 'test block migrations' iptables -I OUTPUT -j REJECT -m state --state NEW -m tcp -p tcp --dport 16509 -m comment --comment 'test block migrations
- Put host H in Maintenance
- Observe that host is indefinitely in PrepareForMaintenance state (after this fix it goes into ErrorInMaintenance after retrying host.maintenance.retries times)
2018-12-28 15:14:16 +05:30
Gerd Müller
cabef5305d server: Enhance bypass vlan overlap check (#3026)
This PR adds the possibility to select a checkbox for the parameter bypassvlanoverlapcheck to the ajax request createNetwork. The checkbox was added for Guest Network as well as for the L2 Guest Network. For L2 Guest Network a backend check for the existence of the flag bypassvlanoverlapcheck was added.
2018-12-24 12:46:54 +05:30
Gabriel Beims Bräscher
bf209405e7 Allow KVM VM live migration with ROOT volume on file storage type (#2997)
* Allow KVM VM live migration with ROOT volume on file

* Allow KVM VM live migration with ROOT volume on file
- Add JUnit tests

* Address reviewers and change some variable names to ease future
implementation (developers can easily guess the name and use
autocomplete)
2018-12-14 09:01:28 -02:00
Andrija Panic
82cdf1c163 db: Add KVM Guest OS mapping for Windows Server 2019 (#3020)
Add KVM Guest OS mapping for Windows Server 2019
2018-11-28 08:53:32 +05:30
Gabriel Beims Bräscher
2334145602
Support requesting a specific IPv4 address (#2595)
This commit allows deploying VMs with a specific IPv4 address.

DirectPodBasedNetworkGuru does not support requesting a custom
IP-Address while creating a new NIC/Instance, throwing the following
error:

    Error 530: Does not support custom ip allocation at this time:
NicProfile[0-0-null-null-null
    Unknown macro: { "cserrorcode"}

Some use-cases prefer the ability to request the IPv4 address which the
Instance will get.

This implementation adds unit test cases to cover and it was manually
tested in Basic Networking. I can perform more tests if requested.
2018-11-27 09:39:10 -02:00
Wido den Hollander
c499be256a ipv6: Advanced Networking Security Groups are supported (#3053)
With earlier work in Basic Networking and the security group provider IPv6 is
supported and we can allow IPv6 to be supplied in networks with SG enabled.

Signed-off-by: Wido den Hollander <wido@widodh.nl>
2018-11-24 15:10:36 +05:30
Paul Angus
fb80e51307 Updating pom.xml version numbers for release 4.11.3.0-SNAPSHOT
Signed-off-by: Paul Angus <paul.angus@shapeblue.com>
2018-11-20 13:11:52 +00:00