CLOUDSTACK-9594: API "list templates templatefilter=all" reveals allAPI "list templates templatefilter=all" reveals all templates.
Using a "list templates templatefilter=all" API call any domain admin can see all templates of all domains in ACS. Information returned includes the account and domain of the template's owner.
The template data shows what that VM is using and any hints from the label. This would give an advantage in what attack vectors to use. The account and domain can possibly be used in brute force attack to guess the password and login information.
Test Scenario:
created two accounts in different domain.
```
mysql> select account_id,username,api_key from user where id in (4,5);
+------------+-----------+----------------------------------------------------------------------------------------+
| account_id | username | api_key |
+------------+-----------+----------------------------------------------------------------------------------------+
| 4 | sudadmin1 | 3qeSuWadNzUFZ_i6c6zbwafjM3Eo0TWpkHw3En9jNsg5Ditk2N18DnbbL2quBYQ7FsdXQ8rwxbyFlE8vyUTwEg |
| 5 | sudadmin | N5uHVOrg1Ek1F1a_5OXTz4WpLG3ewHqcbPUSBjQ-2CTJdxmUe2go0S8fyqH4Np0scYiehYg2KqthZXCWEyKx1A |
+------------+-----------+----------------------------------------------------------------------------------------+
2 rows in set (0.00 sec)
mysql> select account_name,domain_id from account where id in (4,5);
+--------------+-----------+
| account_name | domain_id |
+--------------+-----------+
| sudadmin | 2 |
| sudadmin1 | 3 |
+--------------+-----------+
2 rows in set (0.00 sec)
```
User sudadmin registered a private template named 'Debian'.
http://10.147.59.107:8080/client/api?apikey=N5uHVOrg1Ek1F1a_5OXTz4WpLG3ewHqcbPUSBjQ-2CTJdxmUe2go0S8fyqH4Np0scYiehYg2KqthZXCWEyKx1A&command=listTemplates&templatefilter=self&signature=ODt7zEWCLL20z1FT%2FIkd1molRaM%3D
listTemplate with "templatefilter=self", lists the newly registered template.
```
<listtemplatesresponse cloud-stack-version="4.8.0">
<count>1</count>
<template>
<id>51026d32-60ee-4e25-8ffd-3fa3c57fc14c</id>
<name>Debian</name>
<displaytext>Debian</displaytext>
<ispublic>false</ispublic>
<created>2016-11-10T17:18:00-0500</created>
<isready>true</isready>
<passwordenabled>false</passwordenabled>
<format>VHD</format>
<isfeatured>false</isfeatured>
<crossZones>false</crossZones>
<ostypeid>38c1fc84-a687-11e6-a8c8-06f654000053</ostypeid>
<ostypename>Debian GNU/Linux 7(64-bit)</ostypename>
<account>sudadmin</account>
<zoneid>25fa5b74-d4c2-4bad-8e3a-ceffcd10985e</zoneid>
<zonename>z1</zonename>
<status>Download Complete</status>
<size>2621440000</size>
<templatetype>USER</templatetype>
<hypervisor>XenServer</hypervisor>
<domain>SUDDOMAIN</domain>
<domainid>a350c00d-4048-4876-ae09-74ad4b7bb28c</domainid>
<isextractable>false</isextractable>
<checksum>e87a6d7291b999c92baa9623c9c3c207</checksum>
<details>{hypervisortoolsversion=xenserver61}</details>
<sshkeyenabled>false</sshkeyenabled>
<isdynamicallyscalable>false</isdynamicallyscalable>
</template>
</listtemplatesresponse>
```
User: sudadmin1
listTemplate with "templatefilter=self" does not list any template.
http://10.147.59.107:8080/client/api?apikey=3qeSuWadNzUFZ_i6c6zbwafjM3Eo0TWpkHw3En9jNsg5Ditk2N18DnbbL2quBYQ7FsdXQ8rwxbyFlE8vyUTwEg&command=listTemplates&templatefilter=self&signature=RfKsdg3RxDkqJotbTlHU2RdbdPA%3D
`<listtemplatesresponse cloud-stack-version="4.8.0"/>
`
NO TEMPLATES
**listTemplate with "templatefilter=all" lists all templates**
http://10.147.59.107:8080/client/api?apikey=3qeSuWadNzUFZ_i6c6zbwafjM3Eo0TWpkHw3En9jNsg5Ditk2N18DnbbL2quBYQ7FsdXQ8rwxbyFlE8vyUTwEg&command=listTemplates&templatefilter=all&signature=l5tubfyABT67d1jY702dvtZODbc%3D
Result:
```
<listtemplatesresponse cloud-stack-version="4.8.0">
<count>3</count>
<template>
<id>38451a02-a687-11e6-a8c8-06f654000053</id>
<name>CentOS 5.6(64-bit) no GUI (XenServer)</name>
<displaytext>CentOS 5.6(64-bit) no GUI (XenServer)</displaytext>
<ispublic>true</ispublic>
....
</template>
<template>
<id>51026d32-60ee-4e25-8ffd-3fa3c57fc14c</id>
<name>Debian</name>
<displaytext>Debian</displaytext>
<ispublic>false</ispublic>
<created>2016-11-10T17:18:00-0500</created>
<isready>true</isready>
<passwordenabled>false</passwordenabled>
<format>VHD</format>
<isfeatured>false</isfeatured>
<crossZones>false</crossZones>
<ostypeid>38c1fc84-a687-11e6-a8c8-06f654000053</ostypeid>
<ostypename>Debian GNU/Linux 7(64-bit)</ostypename>
**<account>sudadmin</account>**
<zoneid>25fa5b74-d4c2-4bad-8e3a-ceffcd10985e</zoneid>
<zonename>z1</zonename>
<size>2621440000</size>
<templatetype>USER</templatetype>
<hypervisor>XenServer</hypervisor>
<domain>SUDDOMAIN</domain>
<domainid>a350c00d-4048-4876-ae09-74ad4b7bb28c</domainid>
<isextractable>false</isextractable>
<checksum>e87a6d7291b999c92baa9623c9c3c207</checksum>
<details>{hypervisortoolsversion=xenserver61}</details>
<sshkeyenabled>false</sshkeyenabled>
<isdynamicallyscalable>false</isdynamicallyscalable>
</template>
<template>
<id>5f6af7bb-d965-4b9b-ab45-6d455b0d6bbe</id>
<name>SystemVM Template (XenServer)</name>
<displaytext>SystemVM Template (XenServer)</displaytext>
<ispublic>false</ispublic>
.....
</template>
</listtemplatesresponse>
```
**After Fix:**
http://10.147.59.107:8080/client/api?apikey=3qeSuWadNzUFZ_i6c6zbwafjM3Eo0TWpkHw3En9jNsg5Ditk2N18DnbbL2quBYQ7FsdXQ8rwxbyFlE8vyUTwEg&command=listTemplates&templatefilter=all&signature=l5tubfyABT67d1jY702dvtZODbc%3D
```
<listtemplatesresponse cloud-stack-version="4.8.0">
<count>1</count>
<template>
<id>38451a02-a687-11e6-a8c8-06f654000053</id>
<name>CentOS 5.6(64-bit) no GUI (XenServer)</name>
<displaytext>CentOS 5.6(64-bit) no GUI (XenServer)</displaytext>
<ispublic>true</ispublic>
<created>2016-11-10T09:32:44-0500</created>
<isready>true</isready>
<passwordenabled>false</passwordenabled>
<format>VHD</format>
<isfeatured>true</isfeatured>
<crossZones>true</crossZones>
<ostypeid>38a2bfd6-a687-11e6-a8c8-06f654000053</ostypeid>
<ostypename>CentOS 5.6 (64-bit)</ostypename>
<account>system</account>
<zoneid>25fa5b74-d4c2-4bad-8e3a-ceffcd10985e</zoneid>
<zonename>z1</zonename>
<size>21474836480</size>
<templatetype>BUILTIN</templatetype>
<hypervisor>XenServer</hypervisor>
<domain>ROOT</domain>
<domainid>383e0ea6-a687-11e6-a8c8-06f654000053</domainid>
<isextractable>true</isextractable>
<checksum>905cec879afd9c9d22ecc8036131a180</checksum>
<sshkeyenabled>false</sshkeyenabled>
<isdynamicallyscalable>true</isdynamicallyscalable>
</template>
</listtemplatesresponse>
```
Bug has been fixed considering below points
1. templatefilter=all or isofilter=all is applicable only to admin and domain admin.
2. With templatefilter=all or isofilter=all below are the visiblity of templates in system.
- admin should be able to see all templates/iso in system.
- domain admin should be able to see all public template and templates under its domain tree (including sub domain).
- domain admin in a project context should be able to see all public templates and templates registered
as project account and templates which are shared(using updateTemplatePermission api) with project account.
Also Modified "test/integration/component/test_escalation_listTemplateDomainAdmin.py"
This marvin test was written for this scenario but for the second account "templatefilter=all" is not used.
* pr/1763:
CLOUDSTACK-9594: reverted changes introduced in CLOUDSTACK-9376
CLOUDSTACK-9594: API "list templates templatefilter=all" reveals all templates of all domains
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
templates of all domains
Bug has been fixed considering below points
1. templatefilter=all or isofilter=all is applicable only to admin
and domain admin.
2. With templatefilter=all or isofilter=all below are the visiblity
of templates in system.
a. admin should be able to see all templates/iso in system.
b. domain admin should be able to see all public template and
templates under its domain tree (including sub domain).
c. domain admin in a project context should be able to see all public
templates and templates registered as project account and templates
which are shared(using updateTemplatePermission api) with project account.
Modified
"test/integration/component/test_escalation_listTemplateDomainAdmin.py"
This marvin test is written for this scenario but for the second account
"templatefilter=all" is not used.
* 4.7:
CLOUDSTACK-9376: Restrict listTemplates API with filter=all for root admin
CLOUDSTACK-9369: Restrict default login to ldap/native users
Add lsb-release dependency to mgmt server and agent on Debian/Ubuntu.
Emit template UUID and class type over event bus when deleting templates.
* 4.7:
Removed sleeps and used validateList as requested.
Added required_hardware="false" attr above test_02_root_volume_attach_detach
Modified test_volumes.py to include a hypervisor test for root attach/detach testing
Let hypervisor type KVM and Simulator detach root volumes. Updated test_volumes.py to include a test for detaching and reattaching a root volume from a vm. I also had to update base.py to allow attach_volume to have the parameter deviceid to be passed as needed.
Updated test_volumes.py to include a test for detaching and reattaching a root volume from a vm. I also had to update base.py to allow attach_volume to have the parameter deviceid to be passed as needed.
* 4.6:
CLOUDSTACK-9075 - Uses the same vlan since it should have been already released
CLOUDSTACK-9075 - Adds VPC static routes test
CLOUDSTACK-9075 - Covers Private GW ACL with Redundant VPCs
CLOUDSTACK-9075 - Add method to get list of Physical Networks per zone
CLOUDSTACK-6276 Removing unused parameter in integration test for projects
CLOUDSTACK-6276 Removing unused parameter in integration test
CLOUDSTACK-6276 Fixing affinity groups for projects
CLOUDSTACK-9080: Resource limits for Primary arent respected during attachprimary store resource limit check is not performed while attaching a
volume to a vm. Added them same.
Also added a marvin test case to verify the same.
Testing:
BEFORE
No error is shown in UI when trying to attach a volume even after reaching the resource limits.
```
mysql> select * from resource_limit where type="primary_storage";
+----+-----------+------------+-----------------+-------------+
| id | domain_id | account_id | type | max |
+----+-----------+------------+-----------------+-------------+
| 10 | NULL | 4 | primary_storage | 21474836480 |
+----+-----------+------------+-----------------+-------------+
1 row in set (0.00 sec)
mysql> select * from resource_count where account_id=4 and type='primary_storage';
+----+------------+-----------+-----------------+-------------+
| id | account_id | domain_id | type | count |
+----+------------+-----------+-----------------+-------------+
| 63 | 4 | NULL | primary_storage | 48318382080 |
+----+------------+-----------+-----------------+-------------+
1 row in set (0.00 sec)
```
AFTER
Following error message is shown in UI and the volume is not attached
The resource limits stays the same
```
mysql> select * from resource_limit where type="primary_storage";
+----+-----------+------------+-----------------+-------------+
| id | domain_id | account_id | type | max |
+----+-----------+------------+-----------------+-------------+
| 10 | NULL | 4 | primary_storage | 21474836480 |
+----+-----------+------------+-----------------+-------------+
1 row in set (0.01 sec)
mysql> select * from resource_count where account_id=4 and type='primary_storage';
+----+------------+-----------+-----------------+-------------+
| id | account_id | domain_id | type | count |
+----+------------+-----------+-----------------+-------------+
| 63 | 4 | NULL | primary_storage | 48318382080 |
+----+------------+-----------+-----------------+-------------+
1 row in set (0.00 sec)
```
Marvin test: nosetests --with-marvin --marvin-config=setup/dev/advanced.cfg --zone=xen-zone0 --hypervisor=xenserver test/integration/component/test_ps_resource_limits_volume.py
before the change
```
# do ... === TestName: test_attach_volume_exceeding_primary_limits | Status : FAILED ===
AssertionError: Resource count 23 should match with the expected resource count 22\n
```
After the change
```
# do ... === TestName: test_attach_volume_exceeding_primary_limits | Status : SUCCESS ===
ok
----------------------------------------------------------------------
Ran 1 test in 1178.354s
OK
```
* pr/1107:
CLOUDSTACK-9080: Resource limits for Primary arent respected during attach.
Signed-off-by: Remi Bergsma <github@remi.nl>
primary store resource limit check is not performed while attaching a
volume to a vm. Added them same.
Also added a marvin test case to verify the same.
* 4.6:
CLOUDSTACK-9015 - Delete public IP in order to get both IP and NAT rule removed.
CLOUDSTACK-9015 - Add test to cover the rVPC routers stop/start/reboot scenario
CLOUDSTACK-9015 - Make sure the Backup router can talk to the Master router after a stop/start/reboot
CLOUDSTACK-8925 - Default allow for Egress rules is not being configured properly in VR iptables rulesThis PR fixes the router default policy for egress. When the default is DENY, the router still allows outgoing connections.
The test component/test_routers_network_ops.py was improved to cover that case as well. The results were:
Test redundant router internals ... === TestName: test_01_isolate_network_FW_PF_default_routes_egress_true | Status : SUCCESS ===
ok
Test redundant router internals ... === TestName: test_02_isolate_network_FW_PF_default_routes_egress_false | Status : SUCCESS ===
ok
Test redundant router internals ... === TestName: test_01_RVR_Network_FW_PF_SSH_default_routes_egress_true | Status : SUCCESS ===
ok
Test redundant router internals ... === TestName: test_02_RVR_Network_FW_PF_SSH_default_routes_egress_false | Status : SUCCESS ===
ok
----------------------------------------------------------------------
Ran 4 tests in 3636.656s
OK
/tmp//MarvinLogs/test_routers_network_ops_QDL429/results.txt (END)
* pr/1023:
CLOUDSTACK-8925 - Implement the default egress DENY/ALLOW properly
CLOUDSTACK-8925 - Improve the default egress tests in order to cover newly entered rules
CLOUDSTACK-8925 - Add egress dataset to test_data.py
CLOUDSTACK-8925 - Drop the traffic when default egress is set to false
Signed-off-by: Remi Bergsma <github@remi.nl>
CLOUDSTACK-9007 - Write test to check that the /etc/dhcphosts.txt doesn't contain duplicate IPsThis PR contains a test that will cover the fix on PR #981
The tests does the following:
* Creates account, service offering, network offering, network
* Deploys two virtual machines
- Each machine with a pre-assigned IP
* Creates two FW and PF rules
* Checks that SSH into the VMs works
* Checks default routes from both VMs
* Checks that the /etc/dhcphosts.txt contains 1 entry per VM IP
* Destroys/Expunges 1 VM
* Creates a new VM with the same IP as the destroyed one
* Checks that the /etc/dhcphosts.txt contains 1 entry per VM IP
* pr/1002:
CLOUDSTACK-9007 - Add test check that /etc/dhcphosts.txt doesn't contain duplicate IPs
Signed-off-by: Remi Bergsma <github@remi.nl>
Fix error message in test_isolate_network_FW_PF_default_routesWhile running test_isolate_network_FW_PF_default_routes it is expected
that SSH'ing into a VM does not work immediately. However, when it fails
(as expected) witht he follwoing error
====Trying SSH Connection: Host:192.168.23.12 Uer:root
Port:22 RetryCnt:1===
Traceback (most recent call last):
File "/home/jenkins/workspace/mccloud/mct-run-marvin-tests@2/venv/lib/python2.7/site-packages/marvin/sshClient.py", line 121, in createConnection timeout=self.timeout)
File "/home/jenkins/workspace/mccloud/mct-run-marvin-tests@2/venv/lib/python2.7/site-packages/paramiko/client.py", line 251, in connect retry_on_signal(lambda: sock.connect(addr))
File /home/jenkins/workspace/mccloud/mct-run-marvin-tests@2/venv/lib/python2.7/site-packages/paramiko/util.py", line 270, in retry_on_signal return function()
File /home/jenkins/workspace/mccloud/mct-run-marvin-tests@2/venv/lib/python2.7/site-packages/paramiko/client.py", line 251, in <lambda> retry_on_signal(lambda: sock.connect(addr))
File "/usr/lib64/python2.7/socket.py", line 224, in meth return getattr(self._sock,name)(*args)
error: [Errno 113] No route to host
it would try to print a message that generates a actual error:
```
DEBUG: ====Trying SSH Connection: Host:192.168.23.12 User:root
Port:22 RetryCnt:0===
test_isolate_network_FW_PF_default_routes
(integration.component.test_routers_network_ops.TestIsolatedNetworks):
CRITICAL: EXCEPTION: test_isolate_network_FW_PF_default_routes:
Traceback (most recent call last):,
File "/usr/lib64/python2.7/unittest/case.py", line 369, in run
testMethod()',
File "/home/jenkins/workspace/mccloud/mct-run-marvin-tests@2/test/integration/component/test_routers_network_ops.py", line 448, in test_isolate_network_FW_PF_default_routes self.fail("Failed to SSH into VM - %s" % (public_ip.ipaddress.ipaddress)),
"AttributeError: 'unicode' object has no attribute 'ipaddress'"
```
* pr/972:
Fix error message in test_isolate_network_FW_PF_default_routes
Signed-off-by: Remi Bergsma <github@remi.nl>
While running test_isolate_network_FW_PF_default_routes it is expected
that SSH'ing into a VM does not work immediately. However, when it fails
(as expected) witht he follwoing error
====Trying SSH Connection: Host:192.168.23.12 Uer:root
Port:22 RetryCnt:1===
Traceback (most recent call last):
File "/home/jenkins/workspace/mccloud/mct-run-marvin-tests@2/venv/lib/python2.7/site-packages/marvin/sshClient.py", line 121, in createConnection timeout=self.timeout)
File "/home/jenkins/workspace/mccloud/mct-run-marvin-tests@2/venv/lib/python2.7/site-packages/paramiko/client.py", line 251, in connect retry_on_signal(lambda: sock.connect(addr))
File /home/jenkins/workspace/mccloud/mct-run-marvin-tests@2/venv/lib/python2.7/site-packages/paramiko/util.py", line 270, in retry_on_signal return function()
File /home/jenkins/workspace/mccloud/mct-run-marvin-tests@2/venv/lib/python2.7/site-packages/paramiko/client.py", line 251, in <lambda> retry_on_signal(lambda: sock.connect(addr))
File "/usr/lib64/python2.7/socket.py", line 224, in meth return getattr(self._sock,name)(*args)
error: [Errno 113] No route to host
it would try to print a message that generates a actual error:
DEBUG: ====Trying SSH Connection: Host:192.168.23.12 User:root
Port:22 RetryCnt:0===
test_isolate_network_FW_PF_default_routes
(integration.component.test_routers_network_ops.TestIsolatedNetworks):
CRITICAL: EXCEPTION: test_isolate_network_FW_PF_default_routes:
Traceback (most recent call last):,
File "/usr/lib64/python2.7/unittest/case.py", line 369, in run
testMethod()',
File "/home/jenkins/workspace/mccloud/mct-run-marvin-tests@2/test/integration/component/test_routers_network_ops.py", line 448, in test_isolate_network_FW_PF_default_routes self.fail("Failed to SSH into VM - %s" % (public_ip.ipaddress.ipaddress)),
"AttributeError: 'unicode' object has no attribute 'ipaddress'"
- Do not use the API call because it will read what is in the database, that might not have been updated yet
* Check the status in the router directly instead
- Remove all the sleeps
- It was working before because the Routers were restarting about 10 times for each operation
e.g. adding a VM to a network ot acquiring a new IP.
- Adding stat_rules of internal LB to iptables
We needed one extra rule in the INPUT chain
- With the keepalived fixed they should not be needed anymore. So first reducing them drasticaly
- I am now making a backup of the template file, write to the template file and compare it with the existing configuration
- The template file is recovered afer the process
- I also check if the process is running
- I fixed a bug in the compare method
- I am now updating the configuration variable once the file content is flushed to disk
