This commit implements Ingress and Egress filtering for IPv6 in
Basic Networking.
It allows for opening and closing ports just as can be done with IPv4.
Rules have to be specified twice, once for IPv4 and once for IPv6, for
example:
- 22 until 22: 0.0.0.0/0
- 22 until 22: ::/0
Egress filtering works the same as with IPv4. When no rule is applied all
traffic is allowed. Otherwise only the specified traffic (with DNS being
the exception) is allowed.
Signed-off-by: Wido den Hollander <wido@widodh.nl>
This commit implements basic Security Grouping for KVM in
Basic Networking.
It does not implement full Security Grouping yet, but it does:
- Prevent IP-Address source spoofing
- Allow DHCPv6 clients, but disallow DHCPv6 servers
- Disallow Instances to send out Router Advertisements
The Security Grouping allows ICMPv6 packets as described by RFC4890
as they are essential for IPv6 connectivity.
Following RFC4890 it allows:
- Router Solicitations
- Router Advertisements (incoming only)
- Neighbor Advertisements
- Neighbor Solicitations
- Packet Too Big
- Time Exceeded
- Destination Unreachable
- Parameter Problem
- Echo Request
ICMPv6 is a essential part of IPv6, without it connectivity will break or be very
unreliable.
For now it allows any UDP and TCP packet to be send in to the Instance which
effectively opens up the firewall completely.
Future commits will implement Security Grouping further which allows controlling UDP and TCP
ports for IPv6 like can be done with IPv4.
Regardless of the egress filtering (which can't be done yet) it will always allow outbound DNS
to port 53 over UDP or TCP.
Signed-off-by: Wido den Hollander <wido@widodh.nl>
Use separate lvcreate command on XenServer7 hosts, that checks and passes
different parameters based on the xenserver release version.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Allow DNS queries over TCP when egress filtering is configured.
When using DNSSEC more and more queries are done over TCP and this
requires 53/TCP to be allowed.
Signed-off-by: Wido den Hollander <wido@widodh.nl>
We use some JSP file just for translation of strings in the UI. This is
achievable purely in JavaScript. This removes those JSPs, simplifies
translation usage and workflow (purely JS based). The l10n js (dictionary)
files are generated from existing messages.properties files during client-ui
code generation phase.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
The old commands.properties format included the full class name such as:
createAccount=com.cloud.api.commands.CreateAccountCmd;1
The migration script did not consider this format and fails. With this fix
the migration script will process both the formats, including processing a
commands.properties file with mixed format, for example:
$ cat commands.properties
### Account commands
createAccount=1
deleteAccount=2
markDefaultZoneForAccount=com.cloud.api.commands.MarkDefaultZoneForAccountCmd;3
$ python scripts/util/migrate-dynamicroles.py -d -f commands.properties
Apache CloudStack Role Permission Migration Tool
(c) Apache CloudStack Authors and the ASF, under the Apache License, Version 2.0
Running this migration tool will remove any default-role permissions from cloud.role_permissions. Do you want to continue? [y/N]y
The commands.properties file has been deprecated and moved at: commands.properties.deprecated
Running SQL query: DELETE FROM `cloud`.`role_permissions` WHERE `role_id` in (1,2,3,4);
Running SQL query: INSERT INTO `cloud`.`role_permissions` (`uuid`, `role_id`, `rule`, `permission`, `sort_order`) values (UUID(), 1, '*', 'ALLOW', 0);
Running SQL query: INSERT INTO `cloud`.`role_permissions` (`uuid`, `role_id`, `rule`, `permission`, `sort_order`) values (UUID(), 2, 'deleteAccount', 'ALLOW', 0);
Running SQL query: INSERT INTO `cloud`.`role_permissions` (`uuid`, `role_id`, `rule`, `permission`, `sort_order`) values (UUID(), 2, 'markDefaultZoneForAccount', 'ALLOW', 1);
Static role permissions from commands.properties have been migrated into the db
Running SQL query: UPDATE `cloud`.`configuration` SET value='true' where name='dynamic.apichecker.enabled'
Dynamic role based API checker has been enabled!
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
more detailed error if host file not found or cannot be opened
using mkstemp and mkdtemp for improved security
improve resource cleanup in error conditions in unit test
As requested here: https://github.com/apache/cloudstack/pull/1495
No scripts are using perl so that install requirement can be removed.
The new scripts are using standard python packages only.
Includes extensive unit test.
Due to PR #1054 this patch fixes the dynamic-roles migration script
to use the mysql-connector-python dependency.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
writeIfNotHere requires an array of strings, not a string
* pr/1456:
writeIfNotHere requires an array of strings, not a string
Signed-off-by: Will Stevens <williamstevens@gmail.com>
- Removes commands.properties file
- Fixes apidocs and marvin to be independent of commands.properties usage
- Removes bundling of commands.properties in deb/rpm packaging
- Removes file references across codebase
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
This feature allows root administrators to define new roles and associate API
permissions to them.
A limited form of role-based access control for the CloudStack management server
API is provided through a properties file, commands.properties, embedded in the
WAR distribution. Therefore, customizing API permissions requires unpacking the
distribution and modifying this file consistently on all servers. The old system
also does not permit the specification of additional roles.
FS:
https://cwiki.apache.org/confluence/display/CLOUDSTACK/Dynamic+Role+Based+API+Access+Checker+for+CloudStack
DB-Backed Dynamic Role Based API Access Checker for CloudStack brings following
changes, features and use-cases:
- Moves the API access definitions from commands.properties to the mgmt server DB
- Allows defining custom roles (such as a read-only ROOT admin) beyond the
current set of four (4) roles
- All roles will resolve to one of the four known roles types (Admin, Resource
Admin, Domain Admin and User) which maintains this association by requiring
all new defined roles to specify a role type.
- Allows changes to roles and API permissions per role at runtime including additions or
removal of roles and/or modifications of permissions, without the need
of restarting management server(s)
Upgrade/installation notes:
- The feature will be enabled by default for new installations, existing
deployments will continue to use the older static role based api access checker
with an option to enable this feature
- During fresh installation or upgrade, the upgrade paths will add four default
roles based on the four default role types
- For ease of migration, at the time of upgrade commands.properties will be used
to add existing set of permissions to the default roles. cloud.account
will have a new role_id column which will be populated based on default roles
as well
Dynamic-roles migration tool: scripts/util/migrate-dynamicroles.py
- Allows admins to migrate to the dynamic role based checker at a future date
- Performs a harder one-way migrate and update
- Migrates rules from existing commands.properties file into db and deprecates it
- Enables an internal hidden switch to enable dynamic role based checker feature
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
kvm: Aqcuire lock when running security group Python scriptIt could happen that when multiple instances are starting at the same
time on a KVM host the Agent spawns multiple instances of security_group.py
which both try to modify iptables/ebtables rules.
This fails with on of the two processes failing.
The instance is still started, but it doesn't have any IP connectivity due
to the failed programming of the security groups.
This modification lets the script aqcuire a exclusive lock on a file so that
only one instance of the scripts talks to iptables/ebtables at once.
Other instances of the script which start will poll every 500ms if they can
obtain the lock and otherwise execute anyway after 15 seconds.
* pr/1408:
kvm: Aqcuire lock when running security group Python script
Signed-off-by: Will Stevens <williamstevens@gmail.com>
It could happen that when multiple instances are starting at the same
time on a KVM host the Agent spawns multiple instances of security_group.py
which both try to modify iptables/ebtables rules.
This fails with on of the two processes failing.
The instance is still started, but it doesn't have any IP connectivity due
to the failed programming of the security groups.
This modification lets the script aqcuire a exclusive lock on a file so that
only one instance of the scripts talks to iptables/ebtables at once.
Other instances of the script which start will poll every 500ms if they can
obtain the lock and otherwise execute anyway after 15 seconds.
The lock will be released as soon as the script exists, which is usually within
a few hundred ms.
* 4.7:
CLOUDSTACK-9172 Added cross zones check to delete template and iso
Check the existence of 'forceencap' parameter before use
systemvm: set default umask 022 in injectkeys.sh
The default umask of 0022 is set in Ubuntu and other packages. Set the same
in case of CentOS startup scripts. Use umask 022 in the injectkeys.sh script
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
This is a mandatory argument but it was NOT passed which caused the
re-programming of security groups to fail.
Simple fix to just add the argument since the variable is available
there.
* 4.6:
Revert "Change references of people.apache.org to home.apache.org in the test code"
Change references of people.apache.org to home.apache.org in the test code This closes#1123 Signed-off-by: SrikanteswaraRao Talluri <talluri@apache.org>
CLOUDSTACK-9077 Fix injectkeys.sh to work on CentOS7
CLOUDSTACK-9065: fix bug when creating packaging with noredist flag
The S3 implementation is far from finished, this commit focusses on the bases.
- Upgrade AWS SDK to latest version.
- Rewrite S3 Template downloader.
- Rewrite S3Utils utility class.
- Improve addImageStoreS3 API command.
- Split various classes for convenience.
- Various minor improvements and code optimalisations.
A side effect of the new AWS SDK is that it, by default, uses the V4 signature. Therefore I added an option to specify the Signer, so it stays compatible with previous versions.
CLOUDSTACK-9029: Proper support to identify CentOS 7 version numberhttps://issues.apache.org/jira/browse/CLOUDSTACK-9029
* pr/1033:
CLOUDSTACK-9029: Proper support to identify CentOS 7 version number
Signed-off-by: Remi Bergsma <github@remi.nl>
To configure firewall rules, CloudStack modifies `/etc/sysctl.conf` and
execute those modifications. This may be harmful for several reasons:
1. `/etc/sysctl.conf` may be managed by some configuration management
system. Such a system will constantly restore the previous version.
2. `/etc/sysctl.conf` may contain additional properties that have been
changed later by some system administrator (for example, once a
firewall has been configured, forwarding may have been activated
while it is disabled in `/etc/sysctl.conf`). Executing the file
again at a later time may disrupt the system.
3. Entries are added again and again. `/etc/sysctl.conf` will contain
the same directives repeated several times.
Using a configuration file is not needed as `sysctl` is able to directly
modify sysctl values with `-w` flag.
Signed-off-by: Vincent Bernat <Vincent.Bernat@exoscale.ch>
Guys, can you review it? things need to be discussed:
(1) this supports KVM/QCOW2 only. Anyone want to implement for other Hypervisor/format ?
(2) The original data volume (on primary storage) will be removed.
(3) The script uses the default timeout in libvirtComputingResource. Do we need to add one in global configuration (like copy.volume.wait or backup.snapshot.wait, create.volume.from.snapshot.wait)
(4) In scripts/storage/qcow2/managesnapshot.sh, I use "qemu-img convert -f qcow2 -O qcow2" to copy the snapshot from secondary to primary (hence there is no base image file), instead of "cp -f", this is because convert is faster than cp in my testing.
* pr/732:
CLOUDSTACK-5863: revert volume snapshot for KVM/QCOW2
Signed-off-by: Wei Zhou <w.zhou@tech.leaseweb.com>
This setting works on CentOS 6 / RHEL 6 but does nothing, as
"cpu" cgroup is not mounted. On CentOS 7 / RHEL 7 systemd does
mount cgroups and "cpu" is co-mounted with "cpuacc". Hence, if
we specify "cpu" then this results in an error because it can
only use them both, or none.
By removing the setting, we rely on the default of qemu, which
is:
cgroup_controllers = ["cpu", "devices", "memory", "blkio", "cpuacct", "net_cls"]
Only if they are really mounted, they will be used. So, this will
work on both version 6 and 7.
The 'fix script' didn't work well, as after a reboot you'd still have qemu
throwing errors. Now we can handle the co-mountedcgroups.
In dev environments, there is no /etc/cloudstack/management/db.properties file
That forces you to specify all parameters on the command line. This commit
sets some defaults, like port 3306, user root and localhost.
When available, it will still get settings from the config file and it will
also allow you to override it on the command line. So it is fully backwards
compatible.
