summaryrefslogtreecommitdiff
path: root/gitrepo.py
AgeCommit message (Collapse)Author
2016-12-13gitrepo: fix handling of binary only patchesAndy Doan
Patches that just change binary files like: https://git.linaro.org/uefi/OpenPlatformPkg.git/commit/?id=7ab4bb34b2464a2491868264bdf2931f2acd6452 will produce a None value. We need to treat this like '' to make the rest of our code sane. Change-Id: Id9d1b7104cd8410709e01e08e81f96a1ed80f451
2016-04-06discard giant patchesAndy Doan
We occasionally see giant patches come in, normally from libvirt releases such as: http://libvirt.org/git/?p=libvirt.git;a=commit;h=0086221cb377c7a1c2df58b8ee5eb8223f989765 Patchwork looks at patches line by line, and this consumes tons of memory and runs for hours blocking everything. This detects giant patches and safely discards them. Change-Id: I2403941c708c0923bbe143e783f5869da88f880d
2016-03-01gitrepo: add logic to only warn after 4 failuresAndy Doan
We often find upstream repos are periodically unreachable. This allows us to fail a few times before we actually log an error. (Error logs result in cron email so its good to make sure they are real issues) Change-Id: I8f1a0ae19b36fbd1af4ef26b3d68fa85501577c5
2015-11-24Make commands run via cron easier to debugAndy Doan
Code that uses subprocess.* is always a bit weird to manage when called via cron where any output will cause a warning email. This change allows commands to be run in their natural non "quiet" mode, and only dump stdout/stderr when a problem happens or log.DEBUG is enabled. Change-Id: I8d4e83044c3132d287356cb6d3283e99c902b170
2015-11-24Use bare repositoriesAndy Doan
We have unnecessary complication in our code due to the fact we don't use bare repos. Additionally bare repos are much faster for our needs because we do less file I/O. Why was there ever non-bare? I suspect its just historical baggage. I've converted the repos on staging-patches.linaro.org over and things are working much more reliably from cron now. Change-Id: Ie96022b9dcda7ee34ec7c35d9befea6b5f03f8c1
2015-10-09gitrepo: Handle repos with non UTF-8 commitsAndy Doan
Occasionally, the linux kernel has commits with non-utf8 characters. This keeps us from blowing up and should still work well with the patch compare logic we have that has "fuzzy" matching logic. Change-Id: Ia09248f2574b47c68897699c2ec4976f032b275b
2015-07-15make update_patches smarterAndy Doan
update_patches needs similar logic that we had in import_email for marking patches as superseded. This logic is very close to what is currently in linaro_patchmetrics but done in a more pythonic and testable way. This was tested on the current lng-odp repo and found over 100 patches that this change corrects. The test logic includes some patch examples to help start building a nice base to test our algorithms against: v2 of a patch: http://patches.opendataplane.org/patch/1781/ v3 of a patch: http://patches.opendataplane.org/patch/1792/ Change-Id: I3c41c9ad2cab274389a327d6dac266fc0b7f8244
2015-07-15add more fine-grained handling of update-committed-patchesAndy Doan
At times we need to debug this command to see why certain patches get missed. This change allows us to target the new commits for a specific project, or just parse through a specific list of commits in a project. adapted from linaro-patchmetrics: https://git.linaro.org/infrastructure/linaro-patchmetrics.git/commit/54555f4bbee80e79c8694f8c18f1ef02aca9fb36 Change-Id: Idb93721c5d022716282817210c1e5724ac687f8d
2015-07-15change git-repo to return "commit" objects from dulwichAndy Doan
future changes will need to look at the author, committer, and message of a commit. Change-Id: Idbe898befd12df653dbed876f002f03f8a2a50a4
2015-07-09make git-fetch be quietAndy Doan
This was causing cron emails Change-Id: Ib212fbb619abdd30f0c21fd4ad59b923e1471471
2015-07-07create a more sane way to manage logging options from CLIAndy Doan
We need a way to enable debug logging and the current logging setup was inadequate. This provides a common way to add a "--log" option which includes an argparse.Action that can set the logging level if the users specifies. This also changes the default logging level to WARN so that cron won't send emails unless there is a problem. gitrepo.py was updating to make its output adhere to logging settings and not dump debug info the stdout when not needed. Change-Id: I9c43217c5cf93c6f9cd12555733250f5e6d25c6b
2015-07-07update_commited_patches: add a dryrun modeAndy Doan
This makes doing local development and testing much easier. Change-Id: Ic2e921d1095e59e3a023d4f9420a7e0387b2c859
2015-07-07move repository logic to its own moduleAndy Doan
Looking at update-committed-patches in the linaro-patchmetrics file and seeing where this is headed, I think now is the time to split functionality up so its easier to manage. This also improves the logic of managing the last analyzed commit by introducing a process_unchecked_commits method. The result of the change also simplifies/improves some of the logic needed for unit-testing. Change-Id: I1e28f3fe3b27e1afd414939d28acc5eb5f6387cf