1689 Commits

Author SHA1 Message Date
Rajani Karuturi
f0dd5994b4 Merge pull request #1582 from DaanHoogland/CLOUDSTACK-9408
CLOUDSTACK-9408 for the move away from download.cloud.com
2017-04-24 07:57:07 +05:30
Rajani Karuturi
a4dd6bdeeb Merge pull request #1955 from myENA/virtio-scsi
CLOUDSTACK-8239 Add VirtIO SCSI support for KVM hosts
2017-04-20 15:36:34 +05:30
Daan Hoogland
70ef0788c9 CLOUDSTACK-9408: Fix download urls in sql and scripts
This fixes the agreed upon url on download.cloudstack.org in various
sql files and misc scripts.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2017-04-20 12:33:33 +05:30
Daan Hoogland
f07d729086 CLOUDSTACK-9408 for the move away from download.cloud.com
- commented some occurences of cloud.com as being harmless
  * examples
  * identifiers (internal)
 - changed the URL for vhd-util download
 - changed comments from 'cloud.com' to 'Apache CloudStack'
2017-04-20 11:35:36 +05:30
Rajani Karuturi
d96b3cd19c Merge release branch 4.9 to master
* 4.9:
  CLOUDSTACK-9876: Removed test test_01_test_vm_volume_snapshot as we no longer have that restriction and now after fix for CLOUDSTACK-8663 we allow VM and volume snapshots to exist together
2017-04-18 15:17:00 +05:30
Anshul Gangwar
e82a6351a1 CLOUDSTACK-9876: Removed test test_01_test_vm_volume_snapshot as we no longer have that restriction and now after fix for CLOUDSTACK-8663 we allow VM and volume snapshots to exist together 2017-04-17 12:44:44 +05:30
Rajani Karuturi
5c0979fff5 Merge release branch 4.9 to master
* 4.9:
  CLOUDSTACK-9783: Improve metrics view performance
2017-04-06 12:04:55 +05:30
nvazquez
11562dca7f CLOUDSTACK-9854: Fix test_primary_storage test failure due to live migration 2017-03-30 15:50:51 -03:00
Rajani Karuturi
525c45c1e5 Merge pull request #1994 from nvazquez/CLOUDSTACK-9827
CLOUDSTACK-9827: Storage tags stored in multiple placesIssue description: https://issues.apache.org/jira/browse/CLOUDSTACK-9827

### Fixes
- Create Primary Storage: Persist tags into `storage_pool_tags` instead of `storage_pool_details`
- List Storage Tags: Queries `storage_pool_tags` table instead of `storage_tag_view`
- Find Storage Pools by Tags using proper DAO
- Remove storage tags after deleting Primary Storage
- Remove unused `StorageTagDao`, `StorageTagDaoImpl`, `StorageTagVO` and `storage_tag_view`

* pr/1994:
  CLOUDSTACK-9827: Storage tags stored in multiple places

Signed-off-by: Rajani Karuturi <rajani.karuturi@accelerite.com>
2017-03-28 11:09:56 +05:30
nvazquez
edf0e2b26f CLOUDSTACK-9827: Storage tags stored in multiple places 2017-03-24 13:37:04 -03:00
Nathan Johnson
5c476492b1 CLOUDSTACK-8239 - Adding support for virtio-scsi on KVM hosts
This adds support for virtio-scsi on KVM hosts, either
for guests that are associated with a new os_type of 'Other PV Virtio-SCSI (64-bit)',
or when a VM or template is regstered with a detail parameter rootDiskController=scsi.

Update cloudstack add template dialog to allow for selecting rootDiskController with KVM

Update cloudstack kvm virtio-scsi to enable discard=unmap
2017-03-12 10:54:43 -05:00
nvazquez
c66df6e11f Fix for test failure 2017-03-02 16:14:45 -03:00
nvazquez
b792df163a Fix for test_snapshots.py using nfs2 instead of nfs template 2017-03-01 12:18:35 -03:00
Rohit Yadav
402253504e CLOUDSTACK-9783: Improve metrics view performance
This improves the metrics view feature by improving the rendering performance
of metrics view tables, by reimplementing the logic at the backend and data
served via APIs. In large environments, the older implementation would
make several API calls that increases both network and database load.

List of APIs introduced for improving the performance:

    listClustersMetrics
    listHostsMetrics
    listInfrastructure
    listStoragePoolsMetrics
    listVMsMetrics
    listVolumesMetrics
    listZonesMetrics

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2017-02-28 13:45:03 +05:30
Rajani Karuturi
248c3f72c7 Merge release branch 4.9 to master
* 4.9:
  CLOUDSTACK-9691: Added test list_snapshots_with_removed_data_store
  CLOUDSTACK-9691: Fixed unhandeled excetion in list snapshot command when a primary store is deleted related to it
2017-02-20 16:28:15 +05:30
Nicolas Vazquez
3caedb9ef3 CLOUDSTACK-9691: Added test list_snapshots_with_removed_data_store 2017-02-20 11:30:30 +05:30
nvazquez
3a6d98289c CLOUDSTACK-9539: Support changing Service offering for instance with VM Snapshots 2017-02-16 10:49:09 -03:00
Rajani Karuturi
13bfdd71e6 Merge pull request #1741 from swill/strongswanvpn
Updated StrongSwan VPN ImplementationThis PR is a merge of @jayapalu changes in #872 and the changes I had to make to get the functionality working.

I have done pretty extensive testing of this code so far and we are looking to be in pretty good shape.  One thing to note is that a `Diffie-Hellman` group **is required** in order for this feature to work correctly.  It is not highlighted in the tests below, but I have shown that the `PFS` is not required for this feature to work.  In #872 I have shown a more exhaustive set of tests of this code, but I have limited this set of tests to a recommended `IKE` and `ESP` configuration in order to reduce the noise and test the other areas of functionality.

**Test Results**
I am testing this functionality by creating two VPCs with VMs in each and creating a S2S VPN connection between the two VPCs. Then I SSH into a VM in one VPC and I ping the private IP of a VM in the other VPC. Then I tear it down and try a different configuration.

_Setup_

```
VPC 1                          VPC 2
=====                          =====
VPN Gateway                    VPN Gateway
VPN Customer Gateway           VPN Customer Gateway
VPN Connection        <--->    VPN Connection
 - Passive = True               - Passive = False
```

_Legend_
`SKIP` => At least one of the VPN Connections did not come up, so no test was run.
`OK` => The ping test was successful over the S2S VPN connection.
`FAIL` => The ping test failed over the S2S VPN connection.

`Passive` => Specifies if either the `<vpc_1> : <vpc_2>` sides of the VPN Connection is set to passive.
`Conn State` => Specifies the connection status of the `<vpc_1> : <vpc_2>` VPN Connection in the UI.
`Requires Reset` => If the ping test does not result in an `OK`, then a VPN Connection Reset is performed on either `<vpc_1> : <vpc_2>` sides of the VPN Connection based on which side is not showing `Connected`.  The results in the `Status` column is the final result after the reset is performed.

_Results_

```
+--------+----------------------+-------+-------+----------+----------+---------------+-----------------------------+----------------+
| Status | IKE & ESP            | DPD   | Encap | IKE Life | ESP Life | Passive       | Conn State                  | Requires Reset |
+========+======================+=======+=======+==========+==========+===============+=============================+================+
| OK     | aes128-sha1;modp1536 | True  | False | 86400    | 3600     | True : False  | Disconnected : Connected    | False : False  |
+--------+----------------------+-------+-------+----------+----------+---------------+-----------------------------+----------------+
| OK     | aes128-sha1;modp1536 | True  | True  | 86400    | 3600     | True : False  | Disconnected : Connected    | False : False  |
+--------+----------------------+-------+-------+----------+----------+---------------+-----------------------------+----------------+
| OK     | aes128-sha1;modp1536 | True  | False |          | 3600     | True : False  | Disconnected : Connected    | False : False  |
+--------+----------------------+-------+-------+----------+----------+---------------+-----------------------------+----------------+
| OK     | aes128-sha1;modp1536 | True  | False | 86400    |          | True : False  | Disconnected : Connected    | False : False  |
+--------+----------------------+-------+-------+----------+----------+---------------+-----------------------------+----------------+
| OK     | aes128-sha1;modp1536 | True  | False |          |          | True : False  | Disconnected : Connected    | False : False  |
+--------+----------------------+-------+-------+----------+----------+---------------+-----------------------------+----------------+
| OK     | aes128-sha1;modp1536 | True  | False | 86400    | 3600     | False : False | Connected : Connected       | False : False  |
+--------+----------------------+-------+-------+----------+----------+---------------+-----------------------------+----------------+
| OK     | aes128-sha1;modp1536 | True  | False | 86400    | 3600     | True : True   | Disconnected : Disconnected | False : False  |
+--------+----------------------+-------+-------+----------+----------+---------------+-----------------------------+----------------+
| OK     | aes128-sha1;modp1536 | True  | False | 86400    | 3600     | False : True  | Connected : Disconnected    | False : False  |
+--------+----------------------+-------+-------+----------+----------+---------------+-----------------------------+----------------+
| OK     | aes128-sha1;modp1536 | False | False | 86400    | 3600     | False : False | Connected : Connected       | False : False  |
+--------+----------------------+-------+-------+----------+----------+---------------+-----------------------------+----------------+
| OK     | aes128-sha1;modp1536 | False | False | 86400    | 3600     | True : False  | Disconnected : Connected    | False : False  |
+--------+----------------------+-------+-------+----------+----------+---------------+-----------------------------+----------------+
| OK     | aes128-sha1;modp1536 | False | False | 86400    | 3600     | True : True   | Disconnected : Disconnected | False : False  |
+--------+----------------------+-------+-------+----------+----------+---------------+-----------------------------+----------------+
| OK     | aes128-sha1;modp1536 | False | False | 86400    | 3600     | False : True  | Connected : Disconnected    | False : False  |
+--------+----------------------+-------+-------+----------+----------+---------------+-----------------------------+----------------+
| SKIP   | aes128-sha1          | True  | False | 86400    | 3600     | True : False  | Disconnected : Error        | True : False   |
+--------+----------------------+-------+-------+----------+----------+---------------+-----------------------------+----------------+
| SKIP   | aes128-sha1          | False | False | 86400    | 3600     | True : False  | Disconnected : Error        | True : False   |
+--------+----------------------+-------+-------+----------+----------+---------------+-----------------------------+----------------+
| FAIL   | aes128-sha1          | True  | False | 86400    | 3600     | True : True   | Disconnected : Disconnected | True : True    |
+--------+----------------------+-------+-------+----------+----------+---------------+-----------------------------+----------------+
| SKIP   | aes128-sha1          | True  | False | 86400    | 3600     | False : False | Connected : Error           | False : False  |
+--------+----------------------+-------+-------+----------+----------+---------------+-----------------------------+----------------+
```

* pr/1741:
  complete implementation of the StrongSwan VPN feature

Signed-off-by: Rajani Karuturi <rajani.karuturi@accelerite.com>
2017-02-14 17:59:17 +05:30
Rajani Karuturi
8efdcfc1cd Merge pull request #1183 from sanju1010/tcpports
Marvin test to verify that adding TCP ports 500,4500 and 1701 in vpn should not failPlease refer to JIRA ticket for more details
https://issues.apache.org/jira/browse/CLOUDSTACK-9117

Following is the result info:
Test to add TCP Port Forwarding rule for specific ports(500,1701 and 4500) in VPN ... === TestName: test_08_add_TCP_PF_Rule_In_VPN | Status : SUCCESS ===
ok

---

Ran 1 test in 166.799s

OK

* pr/1183:
  Marvin test to verify that adding TCP ports 500,4500 and 1701 in vpn should not fail Bug-Id: CS-43653 Reviewed-by: Self

Signed-off-by: Rajani Karuturi <rajani.karuturi@accelerite.com>
2017-02-13 16:07:28 +05:30
Rajani Karuturi
a79c1f6722 Merge pull request #1416 from pritisarap12/CLOUDSTACK-8717-Failed-to-start-instance-after-restoring-the-running-instance
CLOUDSTACK-8717: Failed to start instance after restoring the running instance Changing PR title and commit message
In continuation with PR #1411  and  #667

* pr/1416:
  CLOUDSTACK-8717: Failed to start instance after restoring the running instance

Signed-off-by: Rajani Karuturi <rajani.karuturi@accelerite.com>
2017-02-13 16:04:41 +05:30
Will Stevens
f045d65b90 complete implementation of the StrongSwan VPN feature 2017-02-02 16:18:06 -05:00
Rajani Karuturi
7233ac37cd Merge pull request #977 from ustcweizhou/vm-snapshot
[4.10] CLOUDSTACK-8746: VM Snapshotting implementation for KVM

* pr/977:
  Fixes for testing VM Snapshots on KVM. Related to PR 977
  CLOUDSTACK-8746: vm snapshot implementation for KVM

Signed-off-by: Rajani Karuturi <rajani.karuturi@accelerite.com>
2017-01-31 05:58:56 +05:30
Rajani Karuturi
4721c53ea0 Merge pull request #1749 from mike-tutkowski/archived_snapshots
CLOUDSTACK-9619: Updates for SAN-assisted snapshotsThis PR is to address a few issues in #1600 (which was recently merged to master for 4.10).

In StorageSystemDataMotionStrategy.performCopyOfVdi we call getSnapshotDetails. In one such scenario, the source snapshot in question is coming from secondary storage (when we are creating a new volume on managed storage from a snapshot of ours thats on secondary storage).

This usually worked in the regression tests due to a bit of "luck": We retrieve the ID of the snapshot (which is on secondary storage) and then try to pull out its StorageVO object (which is for primary storage). If you happen to have a primary storage that matches the ID (which is the ID of a secondary storage), then getSnapshotDetails populates its Map<String, String> with inapplicable data (that is later ignored) and you dont easily see a problem. However, if you dont have a primary storage that matches that ID (which I didnt today because I had removed that primary storage), then a NullPointerException is thrown.

I have fixed that issue by skipping getSnapshotDetails if the source is coming from secondary storage.

While fixing that, I noticed a couple more problems:

1)       We can invoke grantAccess on a snapshot thats actually on secondary storage (this doesnt amount to much because the VolumeServiceImpl ignores the call when its not for a primary-storage driver).
2)       We can invoke revokeAccess on a snapshot thats actually on secondary storage (this doesnt amount to much because the VolumeServiceImpl ignores the call when its not for a primary-storage driver).

I have corrected those issues, as well.

I then came across one more problem:
         When using a SAN snapshot and copying it to secondary storage or creating a new managed-storage volume from a snapshot of ours on secondary storage, we attach to the SR in the XenServer code, but detach from it in the StorageSystemDataMotionStrategy code (by sending a message to the XenServer code to perform an SR detach). Since we know to detach from the SR after the copy is done, we should detach from the SR in the XenServer code (without that code having to be explicitly called from outside of the XenServer logic).

I went ahead and changed that, as well.

JIRA Ticket:
https://issues.apache.org/jira/browse/CLOUDSTACK-9619

* pr/1749:
  CLOUDSTACK-9619: Updates for SAN-assisted snapshots

Signed-off-by: Rajani Karuturi <rajani.karuturi@accelerite.com>
2017-01-27 05:35:06 +05:30
Simon Weller
6f2a5cf04e Fixes for testing VM Snapshots on KVM. Related to PR 977 2017-01-24 21:48:09 +01:00
Wei Zhou
a2428508e2 CLOUDSTACK-8746: vm snapshot implementation for KVM
(1) add support to create/delete/revert vm snapshots on running vms with QCOW2 format
(2) add new API to create volume snapshot from vm snapshot
(3) delete metadata of vm snapshots before stopping/migrating and recover vm snapshots after starting/migrating
(4) enable deleting of VM snapshot on stopped vm or vm snapshot is not listed in qcow2 image.
(5) enable smoke tests for vmsnaphsots on KVM
2017-01-24 21:47:30 +01:00
Frank Maximus
f246de5e93 Use latest Nuage client.
* CloudStack root pom change to use Amazon WS 11.1.16
  caused our client to fail, as it was depending on classes,
  which are not not present anymore.
  Latest client version uses Gson instead.

* increase robustness of nuagevsp tests
  `- test_nuage_internal_dns - move vm2 creation upwards
  `- test_nuage_static_nat - delete vm in test step to avoid sut restriction

BUG-ID: CLOUDSTACK-9729i
Co-Authored-By: Raf Smeets <raf.smeets@nuagenetworks.net>
Bugfix-for: master
2017-01-24 11:38:59 +01:00
Rohit Yadav
a9f45dfc5f
Merge branch '4.9' 2016-12-23 17:50:42 +05:30
Murali Reddy
b25bb10c59 CLOUDSTACK-9688: Fix VR smoke test failure in vpc_vpn
The test_vpc_vpn uses a cidr that overlaps with the base test environment's
CIDR causing intermittent failure. This changes the cidr to not overlap
with underlying infra and avoid future failures.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-12-23 14:46:36 +05:30
Rohit Yadav
cdbe847d29 CLOUDSTACK-9688: Fix failing test_volumes on centos7/kvm
Due to OS/hypervisor/environmental configuration, detaching a disk/device
using libvirt can be successful without updating the domain configuration (xml).
This leads to reattachment failure as the device is blocked until the next
reboot. This fixes a specific environment case by performing stop/start on
the VM only in case of KVM, which will recreate a fresh domain config (xml)
as KVM VMs have transient domain configs (xmls don't persist).

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-12-22 15:23:00 +05:30
Rohit Yadav
fac8c5fc2a
Merge branch '4.9' 2016-12-09 13:32:43 +05:30
Rohit Yadav
b931b798f8
Merge branch '4.8' into 4.9 2016-12-09 13:32:21 +05:30
Rohit Yadav
ae32aa13ed CLOUDSTACK-9594: Fix regression in test_templates
Fixes regression in component test `test_templates.py`

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-12-09 11:57:40 +05:30
Rohit Yadav
6bb019ab63
Merge branch '4.9' 2016-12-09 00:34:26 +05:30
Rohit Yadav
7c27bf3c90
Merge branch '4.8' into 4.9 2016-12-09 00:33:46 +05:30
Rohit Yadav
8769597523 Merge pull request #1763 from sudhansu7/CLOUDSTACK-9594
CLOUDSTACK-9594: API "list templates templatefilter=all" reveals allAPI "list templates templatefilter=all" reveals all templates.
Using a "list templates templatefilter=all" API call any domain admin can see all templates of all domains in ACS. Information returned includes the account and domain of the template's owner.

The template data shows what that VM is using and any hints from the label. This would give an advantage in what attack vectors to use. The account and domain can possibly be used in brute force attack to guess the password and login information.

Test Scenario:

created two accounts in different domain.

```
mysql> select account_id,username,api_key from user where id in (4,5);
+------------+-----------+----------------------------------------------------------------------------------------+
| account_id | username  | api_key                                                                                |
+------------+-----------+----------------------------------------------------------------------------------------+
|          4 | sudadmin1 | 3qeSuWadNzUFZ_i6c6zbwafjM3Eo0TWpkHw3En9jNsg5Ditk2N18DnbbL2quBYQ7FsdXQ8rwxbyFlE8vyUTwEg |
|          5 | sudadmin  | N5uHVOrg1Ek1F1a_5OXTz4WpLG3ewHqcbPUSBjQ-2CTJdxmUe2go0S8fyqH4Np0scYiehYg2KqthZXCWEyKx1A |
+------------+-----------+----------------------------------------------------------------------------------------+
2 rows in set (0.00 sec)

mysql> select account_name,domain_id from account where id in (4,5);
+--------------+-----------+
| account_name | domain_id |
+--------------+-----------+
| sudadmin     |         2 |
| sudadmin1    |         3 |
+--------------+-----------+
2 rows in set (0.00 sec)
```

User sudadmin registered a private template named 'Debian'.

http://10.147.59.107:8080/client/api?apikey=N5uHVOrg1Ek1F1a_5OXTz4WpLG3ewHqcbPUSBjQ-2CTJdxmUe2go0S8fyqH4Np0scYiehYg2KqthZXCWEyKx1A&command=listTemplates&templatefilter=self&signature=ODt7zEWCLL20z1FT%2FIkd1molRaM%3D

listTemplate with "templatefilter=self", lists the newly registered template.

```
<listtemplatesresponse cloud-stack-version="4.8.0">
<count>1</count>
	<template>
		<id>51026d32-60ee-4e25-8ffd-3fa3c57fc14c</id>
		<name>Debian</name>
		<displaytext>Debian</displaytext>
		<ispublic>false</ispublic>
		<created>2016-11-10T17:18:00-0500</created>
		<isready>true</isready>
		<passwordenabled>false</passwordenabled>
		<format>VHD</format>
		<isfeatured>false</isfeatured>
		<crossZones>false</crossZones>
		<ostypeid>38c1fc84-a687-11e6-a8c8-06f654000053</ostypeid>
		<ostypename>Debian GNU/Linux 7(64-bit)</ostypename>
		<account>sudadmin</account>
		<zoneid>25fa5b74-d4c2-4bad-8e3a-ceffcd10985e</zoneid>
		<zonename>z1</zonename>
		<status>Download Complete</status>
		<size>2621440000</size>
		<templatetype>USER</templatetype>
		<hypervisor>XenServer</hypervisor>
		<domain>SUDDOMAIN</domain>
		<domainid>a350c00d-4048-4876-ae09-74ad4b7bb28c</domainid>
		<isextractable>false</isextractable>
		<checksum>e87a6d7291b999c92baa9623c9c3c207</checksum>
		<details>{hypervisortoolsversion=xenserver61}</details>
		<sshkeyenabled>false</sshkeyenabled>
		<isdynamicallyscalable>false</isdynamicallyscalable>
	</template>
</listtemplatesresponse>

```

User: sudadmin1
listTemplate with "templatefilter=self" does not list any template.

http://10.147.59.107:8080/client/api?apikey=3qeSuWadNzUFZ_i6c6zbwafjM3Eo0TWpkHw3En9jNsg5Ditk2N18DnbbL2quBYQ7FsdXQ8rwxbyFlE8vyUTwEg&command=listTemplates&templatefilter=self&signature=RfKsdg3RxDkqJotbTlHU2RdbdPA%3D

`<listtemplatesresponse cloud-stack-version="4.8.0"/>
`

NO TEMPLATES

**listTemplate with "templatefilter=all" lists all templates**

http://10.147.59.107:8080/client/api?apikey=3qeSuWadNzUFZ_i6c6zbwafjM3Eo0TWpkHw3En9jNsg5Ditk2N18DnbbL2quBYQ7FsdXQ8rwxbyFlE8vyUTwEg&command=listTemplates&templatefilter=all&signature=l5tubfyABT67d1jY702dvtZODbc%3D

Result:

```
<listtemplatesresponse cloud-stack-version="4.8.0">
<count>3</count>
	<template>
		<id>38451a02-a687-11e6-a8c8-06f654000053</id>
		<name>CentOS 5.6(64-bit) no GUI (XenServer)</name>
		<displaytext>CentOS 5.6(64-bit) no GUI (XenServer)</displaytext>
		<ispublic>true</ispublic>
		....
	</template>
	<template>
		<id>51026d32-60ee-4e25-8ffd-3fa3c57fc14c</id>
		<name>Debian</name>
		<displaytext>Debian</displaytext>
		<ispublic>false</ispublic>
		<created>2016-11-10T17:18:00-0500</created>
		<isready>true</isready>
		<passwordenabled>false</passwordenabled>
		<format>VHD</format>
		<isfeatured>false</isfeatured>
		<crossZones>false</crossZones>
		<ostypeid>38c1fc84-a687-11e6-a8c8-06f654000053</ostypeid>
		<ostypename>Debian GNU/Linux 7(64-bit)</ostypename>
		**<account>sudadmin</account>**
		<zoneid>25fa5b74-d4c2-4bad-8e3a-ceffcd10985e</zoneid>
		<zonename>z1</zonename>
		<size>2621440000</size>
		<templatetype>USER</templatetype>
		<hypervisor>XenServer</hypervisor>
		<domain>SUDDOMAIN</domain>
		<domainid>a350c00d-4048-4876-ae09-74ad4b7bb28c</domainid>
		<isextractable>false</isextractable>
		<checksum>e87a6d7291b999c92baa9623c9c3c207</checksum>
		<details>{hypervisortoolsversion=xenserver61}</details>
		<sshkeyenabled>false</sshkeyenabled>
		<isdynamicallyscalable>false</isdynamicallyscalable>
	</template>
	<template>
		<id>5f6af7bb-d965-4b9b-ab45-6d455b0d6bbe</id>
		<name>SystemVM Template (XenServer)</name>
		<displaytext>SystemVM Template (XenServer)</displaytext>
		<ispublic>false</ispublic>
		.....
	</template>
</listtemplatesresponse>

```

**After Fix:**

http://10.147.59.107:8080/client/api?apikey=3qeSuWadNzUFZ_i6c6zbwafjM3Eo0TWpkHw3En9jNsg5Ditk2N18DnbbL2quBYQ7FsdXQ8rwxbyFlE8vyUTwEg&command=listTemplates&templatefilter=all&signature=l5tubfyABT67d1jY702dvtZODbc%3D

```
<listtemplatesresponse cloud-stack-version="4.8.0">
<count>1</count>
	<template>
		<id>38451a02-a687-11e6-a8c8-06f654000053</id>
		<name>CentOS 5.6(64-bit) no GUI (XenServer)</name>
		<displaytext>CentOS 5.6(64-bit) no GUI (XenServer)</displaytext>
		<ispublic>true</ispublic>
		<created>2016-11-10T09:32:44-0500</created>
		<isready>true</isready>
		<passwordenabled>false</passwordenabled>
		<format>VHD</format>
		<isfeatured>true</isfeatured>
		<crossZones>true</crossZones>
		<ostypeid>38a2bfd6-a687-11e6-a8c8-06f654000053</ostypeid>
		<ostypename>CentOS 5.6 (64-bit)</ostypename>
		<account>system</account>
		<zoneid>25fa5b74-d4c2-4bad-8e3a-ceffcd10985e</zoneid>
		<zonename>z1</zonename>
		<size>21474836480</size>
		<templatetype>BUILTIN</templatetype>
		<hypervisor>XenServer</hypervisor>
		<domain>ROOT</domain>
		<domainid>383e0ea6-a687-11e6-a8c8-06f654000053</domainid>
		<isextractable>true</isextractable>
		<checksum>905cec879afd9c9d22ecc8036131a180</checksum>
		<sshkeyenabled>false</sshkeyenabled>
		<isdynamicallyscalable>true</isdynamicallyscalable>
	</template>
</listtemplatesresponse>

```

Bug has been fixed considering below points
1. templatefilter=all or isofilter=all is applicable only to admin and domain admin.
2. With templatefilter=all or isofilter=all below are the visiblity of templates in system.
- admin should be able to see all templates/iso in system.
- domain admin should be able to see all public template and templates under its domain tree (including sub domain).
- domain admin in a project context should be able to see all public templates and templates registered
as project account and templates which are shared(using updateTemplatePermission api) with project account.

Also Modified "test/integration/component/test_escalation_listTemplateDomainAdmin.py"
This marvin test was written for this scenario but for the second account "templatefilter=all" is not used.

* pr/1763:
  CLOUDSTACK-9594: reverted changes  introduced in  CLOUDSTACK-9376
  CLOUDSTACK-9594: API "list templates templatefilter=all" reveals all templates of all domains

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-12-09 00:31:09 +05:30
Rohit Yadav
11dd9fb1ca
Merge branch '4.9' 2016-12-08 17:18:46 +05:30
Rohit Yadav
26a2c9ee9b Merge pull request #1822 from shapeblue/4.9-travis-testfix
CLOUDSTACK-9584: Fix intermittent test failure in `test_volumes`The component/test_volume failures happen when disk offering is random selected to be a custom one. This fixes that.

* pr/1822:
  CLOUDSTACK-9584: Fix intermittent test failure in `test_volumes`

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-12-08 17:17:10 +05:30
Rohit Yadav
f70428a232 Merge pull request #1805 from sudhansu7/CLOUDSTACK-9637
CLOUDSTACK-9637: Template create from snapshot does not populate vm_t**ISSUE**
============
Template create from snapshot does not populate vm_template_details

**REPRO STEPS**
==================
1. Register a template A and specify property:
Root disk controller: scsi
NIC adapter type: E1000
Keyboard type: us

2. Create a vm instance from template A

3. Take volume snapshot for vm instance

4. Delete VM instance

5. Switch to "Storage->Snapshots", convert snapshot to a template B

6. Observe template B does not inherit property from template A, the table vm_template_details is empty

**SOLUTION**: Retrieve and add source template details to VMTemplateVO.

Before Fix:

```

mysql> select id,name,source_template_id from vm_template where id=202;
+-----+--------+--------------------+
| id  | name   | source_template_id |
+-----+--------+--------------------+
| 202 | Debian |               NULL |
+-----+--------+--------------------+
1 row in set (0.00 sec)

mysql> select * from vm_template_details where template_id=202;
+----+-------------+--------------------+-------+---------+
| id | template_id | name               | value | display |
+----+-------------+--------------------+-------+---------+
|  1 |         202 | keyboard           | us    |       1 |
|  2 |         202 | nicAdapter         | E1000 |       1 |
|  3 |         202 | rootDiskController | scsi  |       1 |
+----+-------------+--------------------+-------+---------+
3 rows in set (0.00 sec)

mysql> select id,name,source_template_id from vm_template where source_template_id=202;
+-----+----------------+--------------------+
| id  | name           | source_template_id |
+-----+----------------+--------------------+
| 203 | derived-debian |                202 |
+-----+----------------+--------------------+
1 row in set (0.00 sec)

mysql> select * from vm_template_details where template_id=203;
Empty set (0.00 sec)

After Fix:

mysql> select id,name,source_template_id from vm_template where source_template_id=202;
+-----+--------------------------+--------------------+
| id  | name                     | source_template_id |
+-----+--------------------------+--------------------+
| 203 | derived-debian           |                202 |
| 204 | debian-derived-after-fix |                202 |
+-----+--------------------------+--------------------+
2 rows in set (0.00 sec)

mysql> select * from vm_template_details where template_id=204;
+----+-------------+--------------------+-------+---------+
| id | template_id | name               | value | display |
+----+-------------+--------------------+-------+---------+
|  4 |         204 | keyboard           | us    |       1 |
|  5 |         204 | nicAdapter         | E1000 |       1 |
|  6 |         204 | rootDiskController | scsi  |       1 |
+----+-------------+--------------------+-------+---------+
3 rows in set (0.00 sec)

```

**Marvin Test :** test_template_from_snapshot_with_template_details.py

**Result:**
```
test_01_create_template_snampshot (integration.component.test_template_from_snapshot_with_template_details.TestCreateTemplate) ... === TestName: test_01_create_template_snampshot | Status : SUCCESS ===
ok

----------------------------------------------------------------------
Ran 1 test in 864.523s

OK
```

* pr/1805:
  CLOUDSTACK-9637: Template create from snapshot does not populate vm_template_details

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-12-08 17:14:52 +05:30
Rohit Yadav
6ebf02454b CLOUDSTACK-9584: Fix intermittent test failure in test_volumes
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-12-08 16:25:40 +05:30
Rohit Yadav
0e057ad69e
Merge pull request #1579 from nlivens/nuage_vsp_shared_networks
CLOUDSTACK-9403 : Support for shared networks in Nuage VSP pluginThis is first phase of support of Shared Network in cloudstack through NuageVsp Network Plugin. A shared network is a type of virtual network that is shared between multiple accounts i.e. a shared network can be accessed by virtual machines that belong to many different accounts. This basic functionality will be supported with the below common use case:
- shared network can be used for monitoring purposes. A shared network can be assigned to a domain and can be used for monitoring VMs belonging to all accounts in that domain.

With the current implementation with NuageVsp plugin, Each shared network needs its unique IP address range, and can not overlap with another shared network.
In VSD, it is implemented in below manner:
- In order to have tenant isolation for shared networks, we will have to create a Shared L3 Subnet for each shared network, and instantiate it across the relevant enterprises. A shared network will only exist under an enterprise when it is needed, so when the first VM is spinned under that ACS domain inside that shared network.

PR contents:
1) Support for shared networks with tenant isolation on master with Nuage VSP SDN Plugin.
2) Marvin test coverage for shared networks on master with Nuage VSP SDN Plugin.
3) Enhancements on our exiting Marvin test code (nuagevsp plugins directory).
4) PEP8 & PyFlakes compliance with our Marvin test code.

* pr/1579:
  CLOUDSTACK-9403: Support for shared networks in Nuage VSP plugin

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-12-08 00:06:10 +05:30
Rohit Yadav
6bdc411ff2
Merge branch '4.9' 2016-12-08 00:04:26 +05:30
Frank Maximus
4c91c9c519 CLOUDSTACK-9403: Support for shared networks in Nuage VSP plugin
BUG-ID: CLOUDSTACK-9403
Co-Authored-By: Prashanth Manthena <prashanth.manthena@nuagenetworks.net>
Co-Authored-By: Rahul Singal <rahul.singal@nuagenetworks.net>
Co-Authored-By: Sigert Goeminne <sigert.goeminne@nuagenetworks.net>
Co-Authored-By: Nick Livens <nick.livens@nuagenetworks.net>
2016-12-07 10:03:28 +01:00
Murali Reddy
6749785cab CLOUDSTACK-9339 Virtual Routers don't handle Multiple Public Interfaces correctly
-when processing static nat rule, add a mangle table rule, to mark the traffic
   from the guest vm when it has associated static nat rule so that traffic gets
   routed using the route tabe of the device which has public ip associated

  -fix the case where nic_device_id is empty when ip is getting disassociated
   resulting in empty deviceid in ips.json

  -add utility methods in CsRule, and CsRoute to add 'ip rule' and 'ip route' rules respectivley

  -ensure traffic from all public interfaces are connection marked with device number, and restored
   for the reverse traffic. use the connection marked number to do device specific routing table lookup
   fill the device specific routing table with default route

  -component tests for testing multiple public interfaces of VR
2016-12-07 14:33:24 +05:30
Mike Tutkowski
06806a6523 CLOUDSTACK-9619: Updates for SAN-assisted snapshots 2016-12-06 17:32:56 -07:00
Rohit Yadav
46d11b3711
Merge branch '4.9' 2016-12-05 13:52:22 +05:30
Rohit Yadav
61644f9ea9
Merge branch '4.8' into 4.9 2016-12-05 13:51:43 +05:30
Murali Reddy
db39a06085 CLOUDSTACK-9635: fix test_privategw_acl.py
ensure VLAN used for createPrivateGateway is determined after the guest
networks in the VPC is created, so that we skip VLAN allocated for guest
network for the private network of vpc gateway
2016-12-05 09:54:45 +05:30
Sudhansu
545a214a42 CLOUDSTACK-9637: Template create from snapshot does not populate vm_template_details
Summary: Retrieve and add source template details to VMTemplateVO.
2016-12-03 18:45:08 +05:30
Rohit Yadav
e8de07a1ae
Merge branch '4.9' 2016-12-01 10:38:51 +05:30
Rohit Yadav
5d7288b858
Merge branch '4.8' into 4.9 2016-12-01 10:38:43 +05:30