dynamic-roles: packaging improvementsOn some MySQL server envs, this may cause a SQL statement error, though
I was unable to reproduce it. Since it's not needed, an order by 'sort_order'
is enough, we can safely remove it.
/cc @swill @anshulgangwar @DaanHoogland and others
* pr/1551:
migrate-dynamicroles: use mysql.connector due to #1054
packaging: don't bundle systemvm.zip in rpms
packaging: backup commands.properties as it does not exist in new rpms
dynamic-roles: remove unnecessary order by ID
Signed-off-by: Will Stevens <williamstevens@gmail.com>
CLOUDSTACK-9377: Fix metrics pagesize issueFixes listing of clusters and host to list all clusters/hosts by passing
pagesize=-1.
/cc @koushik-das @swill @DaanHoogland -- fixes the issue raised on dev@ ML by Rashmi
* pr/1540:
CLOUDSTACK-9377: Fix metrics pagesize issue
Signed-off-by: Will Stevens <williamstevens@gmail.com>
SystemVM cleanupsfrom the logrotate docs
> size - With this, the log file is rotated when the specified size is reached. Size may be specified in bytes (default), kilobytes (sizek), or megabytes (sizem).
> Note: If size and time interval options are specified at same time, only size option take effect. it causes log files to be rotated without regard for the last rotation time. If both log size and timestamp of a log file need to be considered by logrotate, the minsize option should be used. logrotate will rotate log file when they grow bigger than minsize, but not before the additionally specified time interval.
* pr/1414:
systemvm, logrotate: remove daily explicitly as it is ignored
Signed-off-by: Will Stevens <williamstevens@gmail.com>
Seems to have a license issue so reverting for now.
This reverts commit 9a20ab8bcbbd39aa012a0ec5a65e66bcc737ee0e, reversing
changes made to 7a0b37a29a8be14011427dcf61bf3ea86e47dbf4.
Due to PR #1054 this patch fixes the dynamic-roles migration script
to use the mysql-connector-python dependency.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Removes bundling of systemvm.zip in cloudstack-common rpms. This is not
done in debian packaging either there we remove for rpms as well, as this
file is not used by any subsystem but systemvm.iso is used.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
In case of rpms, the commands.properties file is bundled at
/usr/share/cloudstack-management/webapps/client/WEB-INF/classes/commands.properties
In case of a rpm upgrade, new rpms won't ship with commands.properties file. For
existing installations this copies the commands.properties file to
/etc/cloudstack/management
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
CLOUDSTACK-9362: Skip VXLANs when rewriting the bridge name for migrations (4.8-2)From the [JIRA issue](https://issues.apache.org/jira/browse/CLOUDSTACK-9362):
> bb8f7c652e
>
> The above commit introduces rewriting of bridge device names when migrating a virtual machine from one host to another. However, it also matches bridges called "brvx-1234" and rewrites them to (in my case) "brem1-1234" - this doesn't match the bridge name on the destination and causes the migration to fail with the error:
>
> error : virNetDevGetMTU:397 : Cannot get interface MTU on 'brem1-1234': No such device
>
> I have flagged this as major because it's not possible to migrate VMs using VXLANs for maintenance, which seems important (it's certainly important to me!).
This is a version of #1508 based against 4.8 (sorry!)
* pr/1513:
Skip VXLANs when rewriting the bridge name for migrations
Signed-off-by: Will Stevens <williamstevens@gmail.com>
* 4.7:
Fix Sync of template.properties in Swift
Configure rVPC for router.redundant.vrrp.interval advert_int setting
Have rVPCs use the router.redundant.vrrp.interval setting
Resolve conflict as forceencap is already in master
Split the cidr lists so we won't hit the iptables-resture limits
Check the existence of 'forceencap' parameter before use
Do not load previous firewall rules as we replace everyhing anyway
Wait for dnsmasq to finish restart
Remove duplicate spaces, and thus duplicate rules.
Restore iptables at once using iptables-restore instead of calling iptables numerous times
Add iptables copnversion script.
Fix Sync of template.properties in SwiftWhen using Swift as a secondary storage, we create a template.properties file which stores the metadata about the template. Currently the data which is present in the file is incorrect which leads to templates becoming unavailable after they are downloaded. This fix makes sure that the template.properties has the correct "path" set so that templates are available.
I've also done a bit of cleanup and made the code bit more clean.
* pr/1331:
Fix Sync of template.properties in Swift
Signed-off-by: Will Stevens <williamstevens@gmail.com>
Reimplement router.redundant.vrrp.interval settingGlobal setting `router.redundant.vrrp.interval` is not used any more and it is now set to a hardcoded 1.
This results in a failover from master->backup when the backup doesn't hear from the master in ~3.6sec. This is a bit too tight, as we've seen failovers during live migrations. We could reproduce it in about half of the cases. Setting this to setting to 2 (tested it by hardcoding it in the systemvms) gives twice as much time and we didn't see issues any more. Instead of updating the hardcoded setting from 1 to 2, I reimplemented the global setting by sending it to the router with the cmd_line, as the non-VPC router also does.
Background:
Why is the maximum failover time in the example 3.6 seconds? This comes from the advertisement interval and the skew time. The default advertisement interval is 1 second (configurable in keepalived.conf). The skew time helps to keep everyone from trying to transition at once. It is a number between 0 and 1, based on the formula (256 - priority) / 256
As defined in the RFC, the backup must receive an advertisement from the master every (3 * advert_int) + skew_time seconds. If it doesn't hear anything from the master, it takes over. With a backup router priority of 100 (as in the example), the failover will happen at most 3.6 seconds after the master goes down.
Source: http://www.hollenback.net/KeepalivedForNetworkReliability
* pr/1486:
Configure rVPC for router.redundant.vrrp.interval advert_int setting
Have rVPCs use the router.redundant.vrrp.interval setting
Signed-off-by: Will Stevens <williamstevens@gmail.com>
Restore iptables at once using iptables-restore instead of calling iptables numerous timesThis makes handling the firewall rules about 50-60 times faster because it is generated in memory and then loaded once. It's work by @borisroman see PR #1400. Reopened it here because I think this is a great improvement.
* pr/1482:
Resolve conflict as forceencap is already in master
Split the cidr lists so we won't hit the iptables-resture limits
Check the existence of 'forceencap' parameter before use
Do not load previous firewall rules as we replace everyhing anyway
Wait for dnsmasq to finish restart
Remove duplicate spaces, and thus duplicate rules.
Restore iptables at once using iptables-restore instead of calling iptables numerous times
Add iptables copnversion script.
Signed-off-by: Will Stevens <williamstevens@gmail.com>
On some MySQL server envs, this may cause a SQL statement error, though
I was unable to reproduce it. Since it's not needed, an order by 'sort_order'
is enough, we can safely remove it.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
This reverts commit 7ce0e10fbcd949375e43535aae168421ecdaa562, reversing
changes made to 29ba71f2db3a3b7dcdad3a7c1f83725cffd56261.
This was reverted because it seemed to be related to an issue
when doing a DeployDC, causing an `addHost` error.
This reverts commit 9f970f28b18534dffe33196ead60ea861f501fa9, reversing
changes made to 6d0c92be7257afcf5f1a9f4453e3470e0083125d.
This was reverted because it seemed to be related to an issue
when doing a DeployDC, causing an `addHost` error.
This reverts commit f88cb880974fa56866492c437af291e40bd1a4f6, reversing
changes made to 688522ecd41d5e2bf09467aa236deb50f7826982.
This was reverted because it seemed to be related to an issue
when doing a DeployDC, causing an `addHost` error.
This reverts commit 540d9572fd491db3ce182d26636fc74ada4e171c.
This was reverted because it seemed to be related to an issue
when doing a DeployDC, causing an `addHost` error.
Add perl-modules as install dependency for cloudstack-agentRequired to run perl scripts that configure networking for VMs.
* pr/1495:
Add perl-modules as install dependency for cloudstack-agent
Signed-off-by: Will Stevens <williamstevens@gmail.com>
DAO: Hit the cache for entity flagged as removed tooI came along this part of the code and I don't see any reason why the cache should not be used when fetching with the "removed" ones. It will help decrease the number of DB queries.
*It can be merged in many CS versions*
* pr/1532:
DAO: Rewrite change for method findByIdIncludingRemoved(ID id)
dao: Hit the cache for entity flagged as removed too since they are put in cache afterwards.
Signed-off-by: Will Stevens <williamstevens@gmail.com>
CPU socket count reporting correctionCPU socket count reporting correction
From https://github.com/MissionCriticalCloud/cosmic-plugin-hypervisor-kvm/pull/16
* pr/1520:
Remove empty spaces causing the build to fail
CPU socket count reporting correction
Signed-off-by: Will Stevens <williamstevens@gmail.com>
Fix Nio/CPU issue and CI failures- Reverts ea2286 that introduced a wakeup on each connection loop run.
- In SSL handshake code removes delegated tasks to be run in separate threads.
/cc @kiwiflyer @swill @jburwell and others for review
@kiwiflyer please help me test this fix and share if it makes the NioConnection robust now, without having the selector consume a lot of CPU. Thanks.
* pr/1543:
test: fix cleanup sequence for test_acl_listvolume test
CLOUDSTACK-9299: Fix test failures on CI
CLOUDSTACK-9348: Make NioConnectio loop less aggressive
Signed-off-by: Will Stevens <williamstevens@gmail.com>
It defaults to 1, which is hardcoded in the template:
./cosmic/cosmic-core/systemvm/patches/debian/config/opt/cloud/templates/keepalived.conf.templ
As non-VPC redundant routers use this setting, I think it makes sense to use it for rVPCs as well.
We also need a change to pickup the cmd_line parameter and use it in the Python code that configures the router.
- Fixes oobm integration test to skip if known ipmitool bug is hit
- Fixes ProcessTest unit test case to use sleep
- Removes redundant unit test that covers code in ProcessTest
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
- Reverts ea2286 that introduced a wakeup on each connection loop run.
- In SSL handshake code removes delegated tasks to be run in separate threads.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Honour GS use_ext_dns and redundant VR VIPThis patch addresses two issues:
On redundant VR setups, the primary resolver being handed out to instances is the guest_ip (primary IP for the VR). This might lead to problems upon failover, at least while the DHCP lease doesn't update (because the primary resolver will be checked first until times out, however it'll be gone upon failover).
If Global Setting use_ext_dns is true, we don't want the VR to be the primary resolver at all.
* pr/1536:
This patch addresses two issues:
Signed-off-by: Will Stevens <williamstevens@gmail.com>
CLOUDSTACK-9334: Support jenv and pyenv to manage Java and Python versions * Adds .java-version to specify Java 1.7 as the JDK version to use
for CloudStack development. jenv does not support comments in
this file. Therefore, no license header is present.
* Adds .python-vresion to specify Python 2.7.11 as the Python to
use for CloudStack development. pyenv does not support comments
in this file. Therefore, no license header is present.
* Adds requirements.txt to specify the default dependencies used
for CloudStack development. It does not include Marvin's
dependencies because it's bundle already defines the dependencies
which are installed with Marvin.
* Updates INSTALL.md to explain how to utilize jenv and pyenv for
CloudStack development
* pr/1460:
CLOUDSTACK-9334: Support jenv and pyenv to manage Java and Python versions
Signed-off-by: Will Stevens <williamstevens@gmail.com>
CLOUDSTACK-8800 : Improved the listVirtualMachines API call to include memory utilization information for a VMThis PR introduces the changes proposed in PR #780 with some work to make the code null safe.
During this PR, I have also removed some unused code.
* pr/1444:
Removed unnecessary check when creating the “userVmResponse” object.
Fixed issues from CLOUDSTACK-8800 that were introduced in PR 780
CLOUDSTACK-8800 : Improved the listVirtualMachines API call to include memory utilization information for a VM for xenserver,kvm and for vmware.
Signed-off-by: Will Stevens <williamstevens@gmail.com>
Remove classes with no referencesI used UCDetector (http://www.ucdetector.org/) as a plugin for Eclipse. With this tool, I discovered a lot of code without any reference (variables, methods and classes).
Following the work that was done at [https://github.com/apache/cloudstack/pull/1448]; this pull request had the goal of removing some of these classes. To check if I wasn't missing anything I searched for any file that could reference some of those classes. As I haven't found any way of these classes being used, they were removed. Note that some of them I found other references, but references such as commented lines or tests, nothing that could indicate their use (as XML files configuring beans or another class instantiating an object with "new").
Waiting for tests. Please tell me if I am missing something.
Removed Classes:
- org.apache.cloudstack.framework.jobs.JobCancellationException (**Note:** removed
variable JobCancellationException in com.cloud.utils.SerialVersionUID)
- org.apache.cloudstack.ldap.NoSuchLdapUserException (**Note:** removed test file
/cloud-plugin-user-authenticator-ldap/test/groovy/org/apache/cloudstack/ldap/NoSuchLdapUserExceptionSpec.groovy)
- com.cloud.agent.api.storage.CreateVolumeOVAAnswer
- com.cloud.exception.MissingParameterValueException
- org.apache.cloudstack.api.response.StatusResponse
- org.apache.cloudstack.api.response.VolumeDetailResponse
- org.apache.cloudstack.api.response.UpgradeVmResponse
- org.apache.cloudstack.api.response.AddIpToVmNicResponse
- org.apache.cloudstack.api.response.TemplateZoneResponse (**Note:** at
org.apache.cloudstack.api.response.TemplateResponse, there is this
comment "To avoid breaking backwards compatibility, we still treat a
template at different zones as different templates, so not embedding
template_zone information in this TemplateZoneResponse set. `private
Set<TemplateZoneResponse> zones;`" but right now it is not used)
- org.apache.cloudstack.api.response.NicDetailResponse
* pr/1453:
Removed classes with no reference
Signed-off-by: Will Stevens <williamstevens@gmail.com>
writeIfNotHere requires an array of strings, not a string
* pr/1456:
writeIfNotHere requires an array of strings, not a string
Signed-off-by: Will Stevens <williamstevens@gmail.com>
Disable NioTestHistorically NioTest has caused issue in CI environment and several developer machines due to network requirements which could be disabled by firewall or security enforcers such as selinux. This disables the test once again using a historic commit 881a6e1
Signed-off-by: Rohit Yadav <rohit.yada
To build and just run this test: mvn clean install -pl utils -Dtest=NioTest
* pr/1538:
utils: Disable NioTest
Signed-off-by: Will Stevens <williamstevens@gmail.com>
CLOUDSTACK-9265 cleanup around httpclient versionssome cleanup done
- replaced HttpStatus from org.apache.commons.httpclient with that from org.apache.http
- removed unthrown HttpException
- left auto reformat in place
* pr/1385:
CLOUDSTACK-9265 cleanup around httpclient versions
Signed-off-by: Will Stevens <williamstevens@gmail.com>
