Think about visual language revealing the action with a few strokes (less data-inc) as clear symbology.
Now the question is how to represent the action/verb and the representation/object.
Import and Export (action)
- file or data "trans-port" as upload or download can be compared to former time save file (disc) and open file (folder opening). In web-based context the visual symbology is rather depicting this as arrows reaching in and out to and from server depicted as box
- "Im" vs "Ex" suggests using arrows in opposing directions, e.g. into the box from upside downwards, out of the box from downside upwards
File formats (representation)
Could vary most and evolve over time. Thus I would exclude representation or objects (here: JPEG, SVG) from generic action-icon itself (here: import/export).
Could add this as separate label near the action-icon.
Thus a button would contain at least 2 elements, optionally a textual description:
[action icon] [object/mode] [textual]
If you use this under tight space restrictions, and only want to show an equally shaped/sized icon, then overlay the icon with a visual label depicting the representation/mode (here: file-format abbreviated as file-extension).
A colored "badge" (as used in Bootstrap CSS) could highlight the different input/output formats by different colors. The color also increases contrast to your underlying monochrome icon. Thus it visually shifts focus to the format while the action remains in background.
Similar question Multiple download buttons for a file type with answers suggests commonly used upload and download icons and their combination with file-formats.