confifu's blog

Week 9 and 10 : GSoC Project Report

Last two weeks I worked on implementing saving and loading of storyboard items and fixed some bugs. For implementing saving and loading I created a copy of the data from the models in KisDocument. That data is kept in sync with the data in models.

Saving and loading of storyboard items are working now. You can save a krita document with storyboards in it and the storyboard data will be saved. Thumbnails are not saved into the .kra file but are loaded using the frame number when the document is loaded. Other than that all data related to the storyboard such as scene name, comments, duration are saved. Since the data is in KisDocument we will have storyboards for each of the .kra files.

I worked on the Export dialog GUI and implemented some of the functions. The user can choose the range of items to render. The layout of the document to be exported can be decided either using the custom options i.e. Rows, Columns and Page Size or it can be specified using an SVG file. On clicking the “Specify layout using SVG file” button the a dialog to choose file would be created to choose the layout file. If the SVG file is selected the custom layout options would be disabled and cannot be changed as they are of no use. On clicking the Export button the user would get to choose the file name and location of the export file.

Other than that I fixed some bugs and changed tests to match new changes made. Also I wrote code documentation for most of the parts implemented till now.

This week I will work on creating a layout for exporting, specifying layout using SVG file and then maybe work on the actual exporting part as well. Other than that I plan to improve on the user documentation based on feedback received.

Week 8: GSoC Project Report

Last week I implemented the duration fields and addition of storyboard items from storyboard. Previously it could only be done from the timeline docker. Also I implemented updating of all affected items’ thumbnail. This makes the docker almost complete sans the capability to save or export.

The duration field is implemented such that any item in the storyboard docker has the duration equal to the next keyframe in any node. This makes sense because the canvas image would be identical to the keyframe image for that duration only, after that the other keyframe’s content would be added to it. Changing duration would move all keyframes in all nodes after the keyframe for that item.

Duration Field on adding a keyframe in between

Changing duration would move all keyframes after the keyframe for that item in all layers. So when we change duration for item with frame 0, all items after it move and so do the keyframes for those items.

Effect of duration field changes on timeline docker

Adding item from storyboard item would add an item after the duration for the current item, current item being the item on which RMB was clicked or the plus icon belongs to. There is an Add Storyboard Item Before option in the right click menu that would add an item just before the current item.

Adding storyboard Item after current item

Adding storyboard item before the current item

Thumbnails for all the affected items would be updated when a keyframe is changed.
I created two classes, KisStoryboardThumbnailRenderScheduler and KisAsyncStoryboardThumbnailRenderer which inherits KisAsyncAnimationRendererBase.

KisStoryboardThumbnailRenderScheduler stores queues of dirty/changed frames. It has two queues. One for the keyframes that are directly changed such as by painting on the canvas. The other queue has frames that have been affected by the changes. The first queue is given preference for rendering. These queues are updated whenever the image becomes idle. After that they are one by one rendered and a signal is emitted that contains the KisPaintDevice. The thumbnails are updated using the KisPaintDevice.

KisAsyncStoryboardThumbnailRenderer renders a single frame in another thread. This class is used by KisStoryboardThumbnailRenderScheduler to render frames.

If we change Layer 2 – frame 7, this would change items for frame 10 and frame 14 as they have keyframes in layer 1 and any changes to layer 2 – frame 7 would change the composite image for frame 10 and 14.

With all the major parts complete we are now ready to get these things tested. This is a shoutout to anyone interested in testing this feature out. If you are on a Linux system you can get the appimage from the following link.
This is the link to the appimage on google drive : link(google drive) (204M)
This is a testing guide that has brief info and directions : link (google doc)
You can leave any feedback or suggestions here or on the Krita-Artists post for this feature. If you come across any bug or crash or something more serious then please inform us at the Phabricator task for this feature.

This week I will look into how Krita saves file and write unit tests for saving and loading.

Week 7: GSoC Project Report

This week I completed unit-tests for interactions between storyboard docker and timeline docker. Also now thumbnails will only be updated when the image is idle, meaning if the image is not painted upon for some time, say a sec, the thumbnail will update. This will improve performance when using the canvas. I also wrote some functions that would help when implementing updating of affected thumbnails.

I wrote unit-tests for the interactions between dockers. Some of these interactions have been implemented and some are yet to be implemented. The planned behavior for various interactions according to tests is :

Items in storyboard will have unique and positive frame numbers and they will always remain sorted by the frame number.
On adding a keyframe in the timeline an item will be added to storyboard docker, if there is no item for that frame already.
On removing a keyframe in the timeline an item will be removed from the storyboard docker if there are no other keyframes at that time in timeline docker.
On moving an item in timeline docker,
- if there is no item for the moved to frame, item is inserted for the “to” keyframe, otherwise not.
- if there is no other keyframe at the moved from frame, item is removed for the “from” keyframe, otherwise not.
Selections in storyboard docker would correspond to the last selected keyframe in timeline docker for which item exists in storyboard docker.
Changing duration in storyboard docker for an item would add hold frames right after the keyframes for the item in timeline docker. If there are multiple layers in an image, hold frames should be added to all the layers.
Changing fps should conserve number of frames, that means if duration for an item was 2s 4f at 24 fps, and then fps changes to 12, then duration would change to 4s 4f.

Now thumbnails would be updated only when the image idle, that means, while the canvas is being painted upon, thumbnails would remain at the last version, and would update only when painting has stopped. It is similar to the overview docker but with a bit less delay.

This week I would work on implementing the remaining interactions and the update of all affected item on keyframe changes.

Week 6: GSoC Project Report

This week I worked on writing tests for interactions between different parts of Krita and investigated how to update all affected items in the storyboard.

I added two test classes, one for interactions between keyframe channels and animation interface and one for the interaction between keyframe channel and the storyboard docker. This was essential because the interaction is broken up into two stages. First the keyframe channel emits signals to the animation interface of the current image when any changes, such as add, remove, move are made. These signals result in emission of similar signals from the animation interface. These final signals can be connected to any part of Krita if it has a pointer to the current image, which dockers have. The first class of tests passes after some changes. I have completed parts of the second test. Some tests are yet to be implemented.

Updating all affected items in the storyboard would cause the canvas to update slowly if we update all items in the same thread. So we would have to use multi-threading for this purpose. There are some classes that already do that, I will use those classes. As this is a non-trivial task I postponed this to the next week.

I faced some problems due to my laptop having some hardware problems, and I had to give it away for repair 😦 I was without a linux machine during the weekends and so couldn’t complete the tests. This week I will complete writing tests, work on implementing update of affected items when keyframe changes and the remaining interactions.

Week 4 and 5: GSoC Project Report

This is the report for week 4 and week 5 combined into one because I couldn’t do much during week 4 due to college tests and assignments, so there was not much to report for that week. These two weeks I worked on implementing interactions between the the storyboard docker and timeline docker (or the new Animation Timeline docker). Most of the interactions from the timeline docker to the storyboard docker are implemented. To list the things implemented:

Selections are synced.
Add, remove and move events in the timeline docker should result in items being added, removed and moved (sensibly).
Thumbnails for the current frame would be updated when you change the contents of that frame.

Selections between storyboard docker and timeline docker are synced. If you select a frame in timeline docker the item corresponding to frame in the storyboard docker would get selected . This works the other way too. So if you select an item in storyboard docker that frame will be selected in the timeline docker.

Adding a keyframe in the timeline docker would add an item for that frame in the storyboard docker, if there is not an item for that frame already.

Removing a keyframe in the timeline docker would remove the item for that frame in the storyboard docker, if there is no other keyframe at that frame.

Moving a keyframe in the timeline docker would move the item for that frame in the storyboard docker so that items in the storyboard docker are always sorted according to frames.

It is possible to not add items to storyboard on addition of keyframe by toggling the lock button in storyboard docker. Removing and moving of keyframes would always cause removal and movement of items in the storyboard docker. So the number of items can be less than number of keyframes, but it can never be more.

Changing a keyframe would update the thumbnail for that frame in the storyboard docker. All affected items are not updated right now, but I am working on it.

This week I would work on updating all affected items when a keyframe is changed and would write unit-tests for the interactions.

Week 3: GSoC Project Report

This week I implemented views, drag and drop of storyboard items in the central view and made some small changes. I also ran unit-tests, checked for memory leaks and debugged code, but unfortunately we couldn’t get it tested by users as we got some crashes.

There are three views to customize what part of the storyboard item you see. Namely they are Thumbnail only, Comments only and Both. This was easy to implement as we only had to make changes to delegate and view class to draw the right parts based on the chosen view.

I also implemented drag and drop of storyboard items. For this I implemented the mimeData and dropMimeData functions and then called the moveRows function in the dropMimeData function to move the rows.

Also we made the add and delete buttons permanent instead of show on hover as their might be some tablet devices that do not support hover.

I made some changes to unit-tests to account for the changes in the design. All the unit-tests passed. I also got some memory leaks that I investigated and fixed using valgrind. But we got some crashes that couldn’t be fixed this week so we couldn’t get the GUI tested by users.

This week I would focus on debugging the code and getting it tested by users.

Week 2: GSoC Project Report

This week I worked on making the UI interactive and configuring the interaction between the comment model and the storyboard model. I also implemented the switching of modes.

The comment model stores the name and visibility of comment fields. It is responsible for the comments menu’s items. Storyboard model’s items have fields to store the contents of each comment field. So whenever a comment is added to the comment model we need to add a child to each storyboard item. Similarly with removing and moving (reordering) of comment items. I connected signals for removing, adding and moving items from the comment model to storyboard model. This signals were used to perform the required actions. Remove and add signals were easy, but qt does not use the moveRows(..) function for drag and drop. Instead it inserts the row to be moved in the desired place and deletes the row. So basically the moving is faked. This results in rowsAdded, dataChanged and rowsRemoved signals. To get the rowsMoved signal I had to reimplement the mimeData and dropMimeData and call moveRows explicitly. Also we must return false in the dropMimeData function otherwise the row at previous position will be deleted as qt assumes the default actions are being followed.

Interaction between comment model and storyboard model.

For drawing the comment boxes we use a QtextDocument and a scrollbar. We store the value to which the item has been scrolled to in a Qt::UserRole field in the index in the model, and use that value to draw the desired part of the text. This makes sure that the data that was visible in the QTextEdit widget while editing is visible when the editor is closed. We also have signals for the scroll bar and its buttons.

There are 3 modes, row, column and grid. Since we have used a QListView it was very easy to implement the three modes. Only the row mode required some extra work as we had to change the orientation of the individual items.

This week I would implement the 3 views, try to get the UI tested by users and run unit tests on the models and debug any problems so far.

Week 1 : GSoC Project Report

This week corresponds to Week 3 in the planned timeline.

I created the MVC classes, namely, StoryboardModel, StoryboardDelegate and StoryboardView and implemented a bare bone GUI without much interactivity.

The StoryboardModel provides an interface to the delegate and view classes to handle the underlying data. The model consists of a list of StoryboardItem objects that correspond to individual storyboard items. Each StoryboardItem object consists of multiple StoryboardChild objects, which store data such as frame number, item name, duration and comment fields. This was set up in this way so as to get a tree based model where the index to a storyboard item is different from the individual data index. Now we can have an index for the storyboard item and also have an index for individual field in it, such as duration or comment field.

The StoryboardDelegate class paints items and handles edit events.

For the StoryboardView class I decided to use a QListView in the beginning, but due to it only handling the parent nodes of the model, I decided to subclass the QListView class. QListView is best suited for this purpose because it has certain functions, such as setFlow(…) and setWrap(…) that can make it very easy to get the results that we are aiming for. setFlow(…) can change the flow of the items from horizontal to vertical and vice-versa. So switching from Column mode to Row mode would be a one-liner. Also we can use setWrap(…) to switch to Grid mode.

This was deceptively time-taking and therefore I couldn’t work too much on making the UI interactive. This week I would get the CommentModel and StoryboardModel to interact so that comments added adds a comment field in each of StoryboardModel’s parent indices. I would implement the view and mode switching, which I think would be easy. Also I would work on making the UI interactive and getting it ready for some basic user testing.

Week 0 – GSoC Project Report

This week corresponds to week 2 of the planned timeline. I had planned to write tests and get started with the MVC classes for the storyboard docker this week. Also the comment menu from previous week was to be implemented.

I managed to setup the comment menu’s delegate and model classes. This would handle the comment section of the main model. It would pass on signals such as toggled visibility, swapping , deletion and addition of comment fields. The comment menu’s model inherits QAbstractListModel. It holds a list of the comment fields such as Action or Dialogue, along with a variable for its visibility state. The visibility variable can be toggled and this should change the visibility of the comment field in the storyboard view.

Setting up the test to build with CMake was a pain, mainly because of my inexperience with CMake. But I managed to build the test and learned some things about CMake.

Writing tests was tougher than I expected since I have no prior experience with writing unit-tests. I wrote test for both models, comment and stroyboard, and their interactions in one test class, mainly because they are supposed to work as a unit. Writing tests gave me a sense of clarity about how I should go about implementing the storyboard model classes. I am using QAbstractItemModelTester from Qt’s test suite to test the model. Also I have implemented some other tests to make sure both the models are in sync, e.g. is adding new comments to commentModel adding columns to the storyboardModel. Also since the QAbstractItemModelTester does not have any destructive testing capabilities, I have tried to cover some of the corner case where the model might fail.

I couldn’t make much progress on implementing the MVC classes for main docker, but I had only planned to start it, so I can cover it in this week.

This week I will focus on getting the MVC classes ready with editable metadata(comment) field.

Week -1 : GSoC Project Report

I have started working on my project earlier due to the uncertain times that we find ourselves in. Hence this is Week -1 report, a week earlier than the official start of coding period. This week corresponds to Week 1 of the planned timeline.

This week was easier than expected. Adding the storyboard docker to Krita’s plugin system was very easy, thanks to the numerous dockers already implemented. Implementing outer GUI was tougher than that, but it was easy on absolute terms. The GUI consists of four QToolButtons namely Export, Comment, Lock(Icon), Arrange(Icon) and a QTableView(which will be promoted to a custom view). Three of these buttons Export, Comment and Arrange have a menu associated with them. Lock is a toggle button.

Export Button’s menu is simple and consists of QActions corresponding to pdf and svg export formats. These buttons would open a Dialog. The dialog is not yet implemented.

Comment button’s menu consists of a QListView and two QToolButtons. The QListView will be drawn based on the comments in main model’s data. This menu is not fully implemented yet. The (…) button is delete button.

Arrange button’s menu consists of 2 button groups Mode and View. The button groups consist of QRadioButtons corresponding to the values offered in Mode and View. This menu is setup so that it stays open after choosing an option, as the user might want to change more than one option. e.g. they might want to change view to column and mode to thumbnail at the same time.

Next week I would try to write unit tests for delegate class, add details to documentation, implement the Storyboard Item class and start implementing the MVC classes.