In-Portal Issue Tracker

Welcome to the In-Portal Open Source CMS Issue Tracker! This is a central management / tracking tool for all types of tasks / issues / bugs for the In-Portal Project. Before reporting any issues, please make sure to read the Guide into Issue Tracker and How to Properly Test and Report Bugs!

Viewing Issue Simple Details Jump to Notes ] Wiki ]  Related Changesets ] View Advanced ] Issue History ] Print ]
ID Category Type Reproducibility Date Submitted Last Update
0001034 [In-Portal CMS] Install / Upgrages task always 2011-04-10 21:21 2011-09-19 16:08
Reporter alex View Status public Project Name In-Portal CMS
Assigned To alex Developer
Priority normal Resolution fixed Fixed in Version 5.1.3-B2
Status closed Product Version Target Version 5.1.3
Time EstimateNo estimate
Summary 0001034: Improving Project Deployment Processes to DEV or LIVE servers
Description The current process of deployment custom project Database Queries which are stored in install_schema.sql/install_data.sql file under custom/install folder is very cumbersome and unfriendly. This is the case since developers have to remember which of the SQLs weren't executed on their database yet. Additionally, it's kind of painful to constantly Import language pack file.

To summarize all this we are looking to have a special OnDeploy event which will perform all of the above tasks and more.

Here is a summary and description of the process:


1. Task(s) "roll-out" to DEV or UAT server process which will happen using special Project Update script

2. As part of this process there is a need to run DB updates, this will be done by triggering DB deployment script which will execute all new SQL statements (described below).

3. All custom (project specific) SQL statements will be stored in custom/install/upgrades.sql file (instead of custom/install/install_data.sql / ...schema.sql) along with the special Comments.

4. Comments will have the following format:

#REV_123(121,123): Adding new section to sub-menu (13324)

where:

- REV_123 - shows DB revision ID (123 is INT set by developer)
- (121,123) - list of other DB revisions that current DB revision is depended on; basically if 121 and 123 revisions are NOT applied and we try applying 123 then some SQL in that revision will fail.
- Adding new section to sub-menu (13324) - a task from In-Business with it's corresponding number so developers know what this DB revision does.

5. 1 Comment will represent 1 DB revision and 1 task in In-Business and can have 1 or more related SQL statements

6. Each DB revision must be UNIQUE across current custom/install/upgrades.sql file. This means that developers ALWAYS must perform an Update on their Working Copy to check if new DB revisions came in and then increase their DB revision numbers accordingly before committing to Repository!

7. There will be a new AppliedDBRevisions field added to the Modules table to store the list of already applied DB revisions. This will make sure the deployment process will NOT run the same SQLs more then once in case if Development Manager need to roll out some particular Tasks and not the entire trunk (latest version of the project)

8. Create special script to deploy language "english.lang" which is part of custom module.



ADDITIONS related to above functionality

9. Currently there is NO way to synchronize what Developer has in AppliedDBRevisions field once he executed he's custom SQLs using phpMyAdmin. In this case Developer would be required to manually do and update that field which is NOT convenient at all. We suggest to add a new section under Admin->Tools->System tools where you can do this by simple pressing Validate DB Revisions button (or similar).

As a result script will Validate current "project_upgrades.sql" script and outline any errors or inconsistencies, and auto-populate all missing DB Revisions from the file into AppliedDBRevisions.

NOTE: Developers should run this before we do Repository Update ONLY.


10. Additionally it would be nice to have the ability to trigger OnDeploy event here too (will update DB structure, reset caches and import Language pack). I would call it something like Deploy DB Update.


I believe we should create a new section on the right side above Templates one and call it Project Deployment and add there these 2 options.

Additional Information
Tags No tags attached.
Reference https://groups.google.com/d/topic/in-portal-dev/Xnc-SFIK1Bk/discussion
Change Log Message Automated project deployment process
Estimate Points 0
Attached Files patch file icon deployment_script_core.patch [^] (1,654 bytes) 2011-04-10 21:21 [Show Content]
patch file icon deployment_script_modules.patch [^] (13,361 bytes) 2011-04-10 21:22 [Show Content]
patch file icon deployment_script_core_v2.patch [^] (9,223 bytes) 2011-04-14 09:46 [Show Content]
patch file icon deployment_script_modules_v2.patch [^] (6,154 bytes) 2011-04-14 09:46 [Show Content]
txt file icon ImprovingProjectDeploymentTrans.txt [^] (1,075 bytes) 2011-06-26 21:13 [Show Content]
patch file icon deployment_script_core_v3.patch [^] (28,457 bytes) 2011-06-28 10:14 [Show Content]
patch file icon deployment_script_modules_v3.patch [^] (5,971 bytes) 2011-06-28 10:18 [Show Content]

- Relationships Relation Graph ] Dependency Graph ]
parent of 0001208closed (5.2.0)alex Improvements to automatic deployment script 
related to 0001278closed (5.2.0)alex HTML not escaped in Deployment script output 

-  Notes
User avatar (0003400)
Dmitry (manager)
2011-04-12 12:09

I have added 9 and 10 describing what we are missing in the process.
User avatar (0003404)
alex (manager)
2011-04-14 09:47

Done.

Also whole deployment code moved to "core" module and now each of the modules can have "install/project_upgrades.sql" file to be processed this way.
User avatar (0003548)
Dmitry (manager)
2011-06-26 21:13
edited on: 2011-06-26 22:47

Several issues found while testing this script:

1. DeploymentHelper class is missing in the Patch v2.

2. custom/install/project_upgrades.sql contain Russian works (should in English if we need this) and some dummy test SQLs (should have no PREFIX or commented our at all)

3. Please use attached file with Translations in your next patch.


Please commit new patches once ready. I'll testing this in Beta 2 and RC1 releases.

User avatar (0003566)
alex (manager)
2011-06-27 04:56

Missing file is bad. I don't have it on my computer too.

Then we need to re-create from patch 1 somehow.
User avatar (0003572)
alex (manager)
2011-06-28 10:15

_v3.patch contains all needed files.
User avatar (0003573)
alex (manager)
2011-06-28 10:15

Tested by developer, since close to release.
User avatar (0003574)
alex (manager)
2011-06-28 10:22

Fix committed to 5.1.x branch. Commit Message:

Fixes 0001034: Improving Project Deployment Processes to DEV or LIVE servers
User avatar (0003821)
alex (manager)
2011-09-19 16:08

Closing, since 5.1.3 release has been released.

- Related Changesets
In-Portal CMS: 5.1.x r14402
Timestamp: 2011-06-28 10:22:48
Author: alex
Details ] Diff ]
Fixes 0001034: Improving Project Deployment Processes to DEV or LIVE servers
mod - /in-portal/branches/5.1.x/core/admin_templates/tools/system_tools.tpl Diff ] File ]
mod - /in-portal/branches/5.1.x/core/install/english.lang Diff ] File ]
mod - /in-portal/branches/5.1.x/core/install/install_data.sql Diff ] File ]
mod - /in-portal/branches/5.1.x/core/install/install_schema.sql Diff ] File ]
mod - /in-portal/branches/5.1.x/core/install/upgrades.sql Diff ] File ]
mod - /in-portal/branches/5.1.x/core/units/admin/admin_events_handler.php Diff ] File ]
add - /in-portal/branches/5.1.x/core/units/helpers/deployment_helper.php File ]
mod - /in-portal/branches/5.1.x/core/units/helpers/helpers_config.php Diff ] File ]
add - /in-portal/branches/5.1.x/tools/run_event.php File ]
Modules :: In-News: 5.1.x r14401
Timestamp: 2011-06-28 10:21:03
Author: alex
Details ] Diff ]
Bug 0001034: Improving Project Deployment Processes to DEV or LIVE servers
mod - /modules/in-news/branches/5.1.x/install/install_data.sql Diff ] File ]
Modules :: In-Link: 5.1.x r14400
Timestamp: 2011-06-28 10:20:50
Author: alex
Details ] Diff ]
Bug 0001034: Improving Project Deployment Processes to DEV or LIVE servers
mod - /modules/in-link/branches/5.1.x/install/install_data.sql Diff ] File ]
Modules :: In-Commerce: 5.1.x r14399
Timestamp: 2011-06-28 10:20:35
Author: alex
Details ] Diff ]
Bug 0001034: Improving Project Deployment Processes to DEV or LIVE servers
mod - /w/in-commerce/branches/5.1.x/install/install_data.sql Diff ] File ]
Modules :: In-Bulletin: 5.1.x r14398
Timestamp: 2011-06-28 10:19:55
Author: alex
Details ] Diff ]
Bug 0001034: Improving Project Deployment Processes to DEV or LIVE servers
mod - /modules/in-bulletin/branches/5.1.x/install/install_data.sql Diff ] File ]
Modules :: Custom: 1.1.x r14396
Timestamp: 2011-06-28 10:19:06
Author: alex
Details ] Diff ]
Bug 0001034: Improving Project Deployment Processes to DEV or LIVE servers
mod - /modules/custom/branches/1.1.x/install/install_data.sql Diff ] File ]
mod - /modules/custom/branches/1.1.x/install/install_schema.sql Diff ] File ]
add - /modules/custom/branches/1.1.x/install/project_upgrades.sql File ]

- Issue History
Date Modified Username Field Change
2012-05-16 08:32 alex Relationship added related to 0001278
2012-02-13 14:40 alex Relationship added parent of 0001208
2011-09-19 16:08 alex Note Added: 0003821
2011-09-19 16:08 alex Status resolved => closed
2011-06-28 10:22 alex Note Added: 0003574
2011-06-28 10:22 alex Status reviewed and tested => resolved
2011-06-28 10:22 alex Fixed in Version => 5.1.3-B2
2011-06-28 10:22 alex Resolution open => fixed
2011-06-28 10:22 alex Assigned To !COMMUNITY => alex
2011-06-28 10:22 alex Changeset attached 5.1.x r14402
2011-06-28 10:21 alex Changeset attached 5.1.x r14401
2011-06-28 10:20 alex Changeset attached 5.1.x r14400
2011-06-28 10:20 alex Changeset attached 5.1.x r14399
2011-06-28 10:19 alex Changeset attached 5.1.x r14398
2011-06-28 10:19 alex Changeset attached 5.1.x r14397
2011-06-28 10:19 alex Changeset attached 1.1.x r14396
2011-06-28 10:18 alex File Added: deployment_script_modules_v3.patch
2011-06-28 10:15 alex Note Added: 0003573
2011-06-28 10:15 alex Status needs testing => reviewed and tested
2011-06-28 10:15 alex Note Added: 0003572
2011-06-28 10:15 alex Assigned To alex => !COMMUNITY
2011-06-28 10:15 alex Status needs work => needs testing
2011-06-28 10:14 alex File Added: deployment_script_core_v3.patch
2011-06-27 04:56 alex Note Added: 0003566
2011-06-26 22:47 Dmitry Note Edited: 0003548 View Revisions
2011-06-26 21:13 Dmitry File Added: ImprovingProjectDeploymentTrans.txt
2011-06-26 21:13 Dmitry Note Added: 0003548
2011-06-26 21:13 Dmitry Assigned To Dmitry => alex
2011-06-26 21:13 Dmitry Status needs testing => needs work
2011-04-14 09:47 alex Note Added: 0003404
2011-04-14 09:47 alex Assigned To alex => Dmitry
2011-04-14 09:47 alex Status needs work => needs testing
2011-04-14 09:46 alex File Added: deployment_script_modules_v2.patch
2011-04-14 09:46 alex File Added: deployment_script_core_v2.patch
2011-04-13 02:14 alex Description Updated View Revisions
2011-04-12 12:09 Dmitry Note Added: 0003400
2011-04-12 12:09 Dmitry Assigned To !COMMUNITY => alex
2011-04-12 12:09 Dmitry Status needs testing => needs work
2011-04-12 12:08 Dmitry Description Updated View Revisions
2011-04-10 21:23 Dmitry Developer Dmitry => alex
2011-04-10 21:23 Dmitry Assigned To => !COMMUNITY
2011-04-10 21:23 Dmitry Developer => Dmitry
2011-04-10 21:23 Dmitry Status active => needs testing
2011-04-10 21:22 Dmitry File Added: deployment_script_modules.patch
2011-04-10 21:22 Dmitry Reference => https://groups.google.com/d/topic/in-portal-dev/Xnc-SFIK1Bk/discussion
2011-04-10 21:22 Dmitry Reporter Dmitry => alex
2011-04-10 21:21 Dmitry New Issue
2011-04-10 21:21 Dmitry File Added: deployment_script_core.patch
2011-04-10 21:21 Dmitry Change Log Message => Automated project deployment process



Web Development by Intechnic
In-Portal Open Source CMS
In-Portal Open Source CMS
Copyright © 2000 - 2009 MantisBT Group

Powered by Mantis Bugtracker