ace issueshttps://gitlab.umiacs.umd.edu/adapt/ace/-/issues2018-01-23T08:32:47-05:00https://gitlab.umiacs.umd.edu/adapt/ace/-/issues/42StatusServlet should display correct depositor totals2018-01-23T08:32:47-05:00Ghost UserStatusServlet should display correct depositor totalsThe current totals shown are based off the paginated versions, and instead should reflect what the depositor actually has.
The way this currently works is by taking the active collections, adding up the total number of files and sizes, ...The current totals shown are based off the paginated versions, and instead should reflect what the depositor actually has.
The way this currently works is by taking the active collections, adding up the total number of files and sizes, then updating the colgroup field. There is a CollectionCountContext which acts as a singleton for collection information, we could do the same for groups (keeping track of a minimal set of information and updating when applicable).
This would look like:
* New ServletContextListener for GroupContext
* SQL + Mapping for our query to get group information
* On init grab all group information
* Update for updating a single groups information
* Might be able to say if a collection has changed, update group... could be tricky though
* Get a bean from the GroupContext for use in the StatusServlet
* Read from this bean when setting group informationhttps://gitlab.umiacs.umd.edu/adapt/ace/-/issues/41Allow truncation of size in Stats Servlet2018-01-23T08:32:47-05:00Ghost UserAllow truncation of size in Stats ServletHave an option to allow one to choose between Bytes/MB/GB/etc in the statistics servletHave an option to allow one to choose between Bytes/MB/GB/etc in the statistics servlethttps://gitlab.umiacs.umd.edu/adapt/ace/-/issues/40Additional Collection States2018-01-23T08:32:47-05:00Ghost UserAdditional Collection StatesIn the event of an audit failure (interrupt, network, etc), we want this information to be easily available to the user. This should be able to be completed without the need for too many changes through the collection state.In the event of an audit failure (interrupt, network, etc), we want this information to be easily available to the user. This should be able to be completed without the need for too many changes through the collection state.1.12https://gitlab.umiacs.umd.edu/adapt/ace/-/issues/39Stats query doesn't work if a collection has no group2018-01-23T08:32:47-05:00Ghost UserStats query doesn't work if a collection has no groupWhen executing the query for the statistics page, an exception is thrown when a collection with no group exists in ace
```
Exception Description: The column result [colgroup] was not found in the results of the query.
Query: ResultSetMap...When executing the query for the statistics page, an exception is thrown when a collection with no group exists in ace
```
Exception Description: The column result [colgroup] was not found in the results of the query.
Query: ResultSetMappingQuery(sql="SELECT l1.date, c.name, c.colgroup, m.count, m.size FROM logevent l1 JOIN ( SELECT collection_id, min(date) AS date FROM logevent WHERE logtype = 20 GROUP BY collection_id) AS l2 ON l1.collection_id = l2.collection_id AND l1.date = l2.date AND l1.logtype = 20 JOIN ( SELECT id, name, colgroup FROM collection ) AS c ON l1.collection_id = c.id JOIN ( SELECT count(id) AS count, sum(size) AS size, parentcollection_id FROM monitored_item GROUP BY parentcollection_id) AS m ON m.parentcollection_id = c.id ")
```https://gitlab.umiacs.umd.edu/adapt/ace/-/issues/38Compare API2018-01-23T08:32:47-05:00Ghost UserCompare APIAdd a rest API mimicking the 'Collection Compare' servlet we have. It taken in something with a data structure like, List<Tuple>, where our Tuple is a (path, hash). The tuple will be a named class, but that can be determined later.
The ...Add a rest API mimicking the 'Collection Compare' servlet we have. It taken in something with a data structure like, List<Tuple>, where our Tuple is a (path, hash). The tuple will be a named class, but that can be determined later.
The response should be similar to the Compare servlet as well, showing items which are the same, different, or not found.1.12https://gitlab.umiacs.umd.edu/adapt/ace/-/issues/36Statistics lazy load2018-01-23T08:32:47-05:00Ghost UserStatistics lazy loadInstead of eagerly loading stats for ALL collections when we open the servlet, let's let a query be formed first, then send the results to the servlet.Instead of eagerly loading stats for ALL collections when we open the servlet, let's let a query be formed first, then send the results to the servlet.1.12https://gitlab.umiacs.umd.edu/adapt/ace/-/issues/35Statistics shows incorrect file count2018-01-23T08:32:47-05:00Ghost UserStatistics shows incorrect file countWhen showing statistics on a collection, a larger number of files is shown than appears in ACE. This is due to both directories and files existing as `MonitoredItem`s. We'll need to update the query with `directory=0`When showing statistics on a collection, a larger number of files is shown than appears in ACE. This is due to both directories and files existing as `MonitoredItem`s. We'll need to update the query with `directory=0`1.12https://gitlab.umiacs.umd.edu/adapt/ace/-/issues/33Fix css in Statistics servlet2018-01-23T08:32:47-05:00Ghost UserFix css in Statistics servletMany of the rows in the table end up going out of bounds due to the length of the date, collection name, and size.Many of the rows in the table end up going out of bounds due to the length of the date, collection name, and size.https://gitlab.umiacs.umd.edu/adapt/ace/-/issues/32REST access for corrupt collections + files2018-01-23T08:32:47-05:00Ghost UserREST access for corrupt collections + filesWe're going to want easy access to see what collections are corrupt, and from those what files are corrupt in a collection. This should be pretty straightforward:
* `/rest/collections` with some various query parameters
* `/rest/collect...We're going to want easy access to see what collections are corrupt, and from those what files are corrupt in a collection. This should be pretty straightforward:
* `/rest/collections` with some various query parameters
* `/rest/collections/{id}/corrupt` or something for viewing fileshttps://gitlab.umiacs.umd.edu/adapt/ace/-/issues/28OOM in AuditThread2018-01-23T08:32:47-05:00Ghost UserOOM in AuditThreadWhen auditing a large collection which has many (500k+) missing files, we pull the entire result set in to memory which then causes us to run out of heap space. We should instead use a cursor to iterate.
This is the result of Monitore...When auditing a large collection which has many (500k+) missing files, we pull the entire result set in to memory which then causes us to run out of heap space. We should instead use a cursor to iterate.
This is the result of MonitoredItemManager.listItemsBefore, of which the only usage is in the AuditThread.https://gitlab.umiacs.umd.edu/adapt/ace/-/issues/7Add role restrictions in ListController2018-01-23T08:32:47-05:00Ghost UserAdd role restrictions in ListControllerCurrently anyone can browse collections and get their status information. We should mimic the webapp's roles for how viewing collections is handled.Currently anyone can browse collections and get their status information. We should mimic the webapp's roles for how viewing collections is handled.https://gitlab.umiacs.umd.edu/adapt/ace/-/issues/47Count incorrect after removing files from collections2018-05-08T14:11:22-04:00Ghost UserCount incorrect after removing files from collectionsWe noticed when removing some files from the production ACE at UMIACS that the collection count was not properly updated. These were removed from the Browse Servlet, so that's the first place to start looking.We noticed when removing some files from the production ACE at UMIACS that the collection count was not properly updated. These were removed from the Browse Servlet, so that's the first place to start looking.https://gitlab.umiacs.umd.edu/adapt/ace/-/issues/3Display If There's a Token Ingestion in ACE2018-05-16T11:46:19-04:00Ghost UserDisplay If There's a Token Ingestion in ACEWhen ingesting tokens for large collections we need feedback on what's going on. Similar to how audits display information both in changing the icon and status, we should have something for token ingestion.When ingesting tokens for large collections we need feedback on what's going on. Similar to how audits display information both in changing the icon and status, we should have something for token ingestion.https://gitlab.umiacs.umd.edu/adapt/ace/-/issues/52Ambiguous log statement2018-05-16T13:29:43-04:00Ghost UserAmbiguous log statementA frequent point of concern from people looking at the `aceam.log` are messages of the form:
```
[26/Apr/2018:15:46:43] [Audit] Driver returned Item: /tagmanifest-sha256.txt error: false error msg: null hash: 748cddb196ac6a1bc36009052573...A frequent point of concern from people looking at the `aceam.log` are messages of the form:
```
[26/Apr/2018:15:46:43] [Audit] Driver returned Item: /tagmanifest-sha256.txt error: false error msg: null hash: 748cddb196ac6a1bc360090525730f3c0774efb6d20463ebe19b661f462da6d2
```
we should include a proper delimiter to separate each key/value in the message.https://gitlab.umiacs.umd.edu/adapt/ace/-/issues/51Single session when removing items2018-05-16T13:30:29-04:00Ghost UserSingle session when removing itemsWhen issuing a remove, a new session is used for each log event which is created. It would be nice to tie together batch removals in a single session so it can be queried easily from the event log.When issuing a remove, a new session is used for each log event which is created. It would be nice to tie together batch removals in a single session so it can be queried easily from the event log.https://gitlab.umiacs.umd.edu/adapt/ace/-/issues/48Update CollectionCountContext when a Collection is updated2018-05-16T13:31:06-04:00Ghost UserUpdate CollectionCountContext when a Collection is updatedThere are various situations where items might be removed/added/modified and the CollectionCountContext does not get properly updated. We should ensure that modifications to a collection will update the respective context:
* fileCountMap...There are various situations where items might be removed/added/modified and the CollectionCountContext does not get properly updated. We should ensure that modifications to a collection will update the respective context:
* fileCountMap
* fileActiveMap
* fileCorruptMap
* fileMissingMap
* fileMissingTokenMap
* fileTokenMismatchMap
* totalErrorMap
* totalSizeMap
* fileRemoteMissing
* fileRemoteCorrupt
Also when working on this issue, now might be a good time to check out a different way of caching these (using caffeine or guava).https://gitlab.umiacs.umd.edu/adapt/ace/-/issues/30AuditTokens ends before validation finishes2018-05-16T14:11:28-04:00Ghost UserAuditTokens ends before validation finishesThe log event `Token Audit Finish` can be seen occurring before `Token Valid` events complete. This is likely due to running tasks for cleaning up the audit token thread before the validator is closed (and is given time to finish).The log event `Token Audit Finish` can be seen occurring before `Token Valid` events complete. This is likely due to running tasks for cleaning up the audit token thread before the validator is closed (and is given time to finish).1.13https://gitlab.umiacs.umd.edu/adapt/ace/-/issues/27Peer Auditing Collection Pulling2018-05-18T15:34:17-04:00Ghost UserPeer Auditing Collection PullingWith the new pagination of collections, the peer auditing selections are restricted and only show a limited view of the peer ACE.
One option would be to choose the group and collection separately. Otherwise we could try to have somethin...With the new pagination of collections, the peer auditing selections are restricted and only show a limited view of the peer ACE.
One option would be to choose the group and collection separately. Otherwise we could try to have something where we continue scrolling, though I'm not sure how that would integrate in to ACE.1.13https://gitlab.umiacs.umd.edu/adapt/ace/-/issues/46Update default IMS port to 802018-05-21T17:06:49-04:00Ghost UserUpdate default IMS port to 80Currently when connecting to the ims the default settings are to use port 8080, which at the moment is down. We should update the settings to use 80 by default as we can always expect to serve it from umiacs.Currently when connecting to the ims the default settings are to use port 8080, which at the moment is down. We should update the settings to use 80 by default as we can always expect to serve it from umiacs.https://gitlab.umiacs.umd.edu/adapt/ace/-/issues/29StatusServlet has filter on collection '0'2018-05-21T17:55:04-04:00Ghost UserStatusServlet has filter on collection '0'There are numerous times when going back to the status servlet that it errantly has `0` as a filter for the collection. This is easiest triggered by viewing the event log for a collection, then clicking `Status` to get back to the main p...There are numerous times when going back to the status servlet that it errantly has `0` as a filter for the collection. This is easiest triggered by viewing the event log for a collection, then clicking `Status` to get back to the main page for ACE.