Using pvrtexlib in multiples threads

This topic contains 10 replies, has 7 voices, and was last updated by  Tobias Hector 4 years, 6 months ago.

Viewing 11 posts - 1 through 11 (of 11 total)
  • Author
    Posts
  • #29821

    flamaros
    Member

    Hi,

    We have made a tool to build textures packadges for our iPhone game RTMI, but I need to improve the format of packadges to reduce the loading time. I have completly rewritten our tool to parallize the building, because it’s really slow.

    I need to compress about 30 000 pictures at 512×512 resolution in pvr format, it takes around 8 hours on a single CPU core, that why I really need to improve that to be able to do my optimisations tests faster.

    I have simply put the packadge building in separate threads, it works fine when I used png format (we use this format at begin to test the render part) instead of pvr.

    When I use many thread I am getting this error message : “An error occurred in the pixel format encoder.”

    Is there a way to compress many pvr files in thread safe mode?

    I can’t use the command line tool beacause we have a border issue. Need I build my own command tool based on the library?

    #33119

    flamaros
    Member

    I me again,

    I try to do an external binary, I have always this issue, it’s not due to my modifications on my tools because when I force it to use only one thread (it’s directly integrated on the tool interface) I don’t have this problem.

    #33120

    flamaros
    Member

    Ok sorry,

    It was my bad I forgot to change my code every where, I’ll clean that.

    #33121

    Gordon
    Moderator

    Have you managed to get this working successfully?

    I have never tried to use PVRTexLib in a multithreaded context and cannot vouch for it being suitable. We have been working on faster PVRTC compression, but I can’t promise when this will become available.

    #33122

    ringo
    Member

    I confirm that the latest PVRTexLib doesn’t work in multithreaded environment 🙁

    It seems you’re using static memory buffer or something during compress process. I’ve tried to compress PVRTC4 textures in parallel and got some bad artifacts in output textures.
    #33123

    Gordon
    Moderator

    Sadly, I’m not very surprised. The existing PVRTC compressor was not built for speed or flexibility.

    However, we have been working on new PVRTC code which does run multi-threaded and is much faster in general for the next major release of the SDK. Please check back for an update.

    #33124

    aras_p
    Member
    Gordon wrote:
    However, we have been working on new PVRTC code which does run multi-threaded and is much faster in general for the next major release of the SDK. Please check back for an update.

    So 1.5 years later… any news on this?

    PVRTC compression times are killing all iOS developers and about 1/3rd of Android developers here at Unity. It takes about 8.5 seconds for one 1024×1024 texture to be compressed into 4bpp PVRTC (Core i7 2.66GHz; compressor uses 1 core). That really breaks iteration, especially as artists try to tweak the textures to avoid compression artifacts.

    #33125

    Hi,

    The multi-threaded processor mentioned is already implemented in PVRTexTool, as the “Fast Compressor” option when compressing. The quality is not quite as good as High Quality compression, but the artifacts tend to be the same between compressors, so it might be best to use fast compression during development, then high quality only for checking near the end. Work on threading the high quality compressor is ongoing, and once it’s done, it should also be a fair bit faster.

    Thanks,

    Tobias

    #33126

    Simon Fenney
    Moderator
    aras_p wrote:
    PVRTC compression times are killing all iOS developers and about 1/3rd of Android developers here at Unity. It takes about 8.5 seconds for one 1024×1024 texture to be compressed into 4bpp PVRTC (Core i7 2.66GHz; compressor uses 1 core). That really breaks iteration, especially as artists try to tweak the textures to avoid compression artifacts.

    Hi Aras,
    As Tobias says, we are working on a new compressor library which does a better job, quality wise, than the current PVRTexturetool. This code will be multithreaded. (Currently, it it uses pthreads but we will adapt it for windows as well) .

    However, because it performs a more thorough exploration of the possible options, the default “best quality” search is currently slightly slower (even allowing for multi-threading) than the PVRtextool. Now, we will be able to add “fast” and “medium” quality compression settings but I’m wondering if that will counter-productive. You say that the artists are doing trial compressions and then tweaking the textures but, if a less thorough compression search is performed in each artist’s edit-compress-view iteration, don’t you run the risk of more artefacts?

    #33127
    aras_p wrote:
    PVRTC compression times are killing all iOS developers and about 1/3rd of Android developers here at Unity. It takes about 8.5 seconds for one 1024×1024 texture to be compressed into 4bpp PVRTC (Core i7 2.66GHz; compressor uses 1 core). That really breaks iteration, especially as artists try to tweak the textures to avoid compression artifacts.

    Hi Aras,
    As Tobias says, we are working on a new compressor library which does a better job, quality wise, than the current PVRTexturetool. This code will be multithreaded. (Currently, it it uses pthreads but we will adapt it for windows as well) .

    However, because it performs a more thorough exploration of the possible options, the default “best quality” search is currently slightly slower (even allowing for multi-threading) than the PVRtextool. Now, we will be able to add “fast” and “medium” quality compression settings but I’m wondering if that will counter-productive. You say that the artists are doing trial compressions and then tweaking the textures but, if a less thorough compression search is performed in each artist’s edit-compress-view iteration, don’t you run the risk of more artefacts?

    *bump* two years on, does this exist? PVRTexTool is really slowing down our build processes.

    #33128

    Hi Metaphysics,

    Yes it was done some time early last year. The current compressor spawns threads to do the heaviest parts of its workload. Unfortunately the times are still slow due to the complexity, but we’re still working on actively developing it.

    Thanks,

    Tobias

Viewing 11 posts - 1 through 11 (of 11 total)
You must be logged in to reply to this topic.