This change removes the timestamp field from messages and conversations
in favor of multiple semantically named timestamp fields: sent_at,
received_at on messages; active_at on conversations. This requires/lets
us rethink and improve our indexing scheme thusly:
The inbox index on conversations will order entries by the
conversation.active_at property, which should only appear on
conversations destined for the inbox.
The receipt index will use the message.sent_at property, for effecient
lookup of outgoing messages by timestamp, for use in processing delivery
receipts.
The group index on conversation.members is multi-entry, meaning that
looking up any phone number in this index will efficiently yield all
groups the number belongs to.
The conversation index lets us scan messages in a single conversation,
in the order they were received (or the reverse order). It is a compound
index on [conversationId, received_at].
This ended up turning into a rewrite/refactor of the background page.
For best results, view this diff with `-w` to ignore whitespace. In
order to support retrying message decryption, possibly at a much later
time than the message is received, we now implement the following:
Each message is saved before it is decrypted. This generates a unique
message_id which is later used to update the database entry with the
message contents, or with any errors generated during processing.
When an IncomingIdentityKeyError occurs, we catch it and save it on the
model, then update the front end as usual. When the user clicks to
accept the new key, the error is replayed, which causes the message to
be decrypted and then passed to the background page for normal
processing.
ReplayableErrors make it easy for the frontend to handle identity key
errors by wrapping the necessary steps into one convenient little
replay() callback function.
The frontend remains agnostic to what those steps are. It just calls
replay() once the user has acknowledged the key change.
The protocol layer is responsible for registering the callbacks needed
by the IncomingIdentityKeyError and OutgoingIdentityKeyError.
superfeedr has done a nice job with this backbone -> indexedDB adapter,
but their query interface is somewhat limited. This commit adds an
alternate interface that lets us specify the index and cursor bounds we
want. This interface requires deeper knowledge of indexedDB indices, but
is more powerful overall.
This was used to conditionally render messages in the group style, but
it's actually unnecessary. We can render the same markup in both cases
and change the appearance with css.
This commit provides the javascript complement to
[WebSocket-Resources](https://github.com/WhisperSystems/WebSocket-Resources),
allowing us to use a bi-directional request-response framework over
websockets.
See websocket-resources.js and websocket-resources_test.js
for usage details.
Along the way I also factored the websocket keepalive and reconnect
logic into its own file/wrapper object.
Move base64 encoding of attachments to an AttachmentView. This makes
image rendering an asynchronous task so we fire an update event to
indicate to the parent MessageListView that its content has changed
height and it is time to scroll down.
Register the runtime callback at the top level view rather than having
each conversation view register independently.
Also refactors Layout into InboxView.
After a message is saved asynchronsly, fire an event and pass the
message attributes to frontend listeners via the chrome-runtime API.
This behavior is similar to the 'storage' event fired by localStorage.
Getting up and running with IndexedDB was pretty easy, thanks to
backbone. The tricky part was making reads and writes asynchronous.
In that process I did some refactoring on Whisper.Threads, which
has been renamed Conversations for consistency with the view names.
This change also adds the unlimitedStorage permission.
Eliminates the global Whisper.Messages object and consolidates shared
send/receive logic in Whisper.Threads.
To the latter end, note that the decrypted array buffer on an attachment
pointer is now named data instead of decrypted, in order to match the
format of outgoing attachments presented by
FileReader.readAsArrayBuffers and let us use the same handler to base64
encode them.
This dependency may be a little heavy for our current use case, but we can
roll with it for now and find something slimmer if it turns out yagni.
Closes#77Closes#40
Runtime reload is overkill and causes a jarring ux. Instead, send and
receive messages across the runtime. Also, if we need to jump between
the main ui and options pages, simply navigate within the current tab
rather than spawning a new one.
We only depend on cryptojs for this webcrypto polyfill, so let Grunt
concatenate them into one file.
The reference in the getString helper isn't needed since we use the
built in string converters on CryptoJS's word arrays.
Rename methods on the curve25519 interface to be a bit more high level.
Cleanup emscripten wrapper class, wrap long lines and such. Also add a
grunt task alias for building the emscripten compiled curve
implementation.
Firstly, don't initialize textsecure.nativclient unless the browser
supports it. The mimetype-check trick is hewn from nacl-common.js.
Secondly, nativeclient crypto functions will all automatically wait for
the module to load before sending messages, so we needn't register any
onload callbacks outside nativeclient.js. (Previously, if you wanted to
do crypto with native client, you would have to register a call back and
wait for the module to load.) Now that the native client crypto is
encapsulated behind a nice interface, it can handle all that
onload-callback jazz internally: if the module isn't loaded when you
call a nativeclient function, return a promise that waits for the load
callback, and eventually resolves with the result of the requested
command. This removes the need for textsecure.registerOnLoadCallback.
Finally, although native client has its quirks, it's significantly
faster than the alternative (emscripten compiled js), so this commit
also lets the crypto backend use native client opportunistically, if
it's available, falling back to js if not, which should make us
compatible with older versions of chrome and chromium.
Man you shoulda been there. Code was compiling all over the damn place.
It was wild.
Seriously though. Ignore that intermediate compiled file. What happens
in build stays in build.
Build with `grunt compile && grunt concat:curve25519` after installing
emscripten.
Enable by either (a) not loading nativeclient.js or (b) setting
`textsecure.NATIVE_CLIENT = false` before loading nativeclient.js.
The nativeclient.js module overrides
window.textsecure.registerOnLoadFunction with its own version. Otherwise
helpers will define a trivial placeholder for same.
The flag textsecure.NATIVE_CLIENT can be set anywhere ahead of
nativeclient.js, but is only acted on in nativeclient.js,
and crypto.js.
NB: this diff is best viewed with --ignore-whitespace
Distills crypto.js down to the hard cryptoey bones. It pulls from
webcrypto for aes and hmac, and from native client for curve25519 stuff
or potentially another object implementing the handful of needed
curve25519 functions.
Everything else formerly known as crypto, including session storage and
management, axolotl, etc.. is now protocol.js. The separation is not
quite perfect, but it's a big step.
nativeclient.js now enables talking to the native client module through
a high level interface as well as registering callbacks that will be
executed once the module is loaded. And it has tests!
Finally, this commit removes all references to the "testing_only"
object, preferring to run tests on textsecure.crypto instead.
To components. Because tab-completion works better when there aren't two
things starting with bower, and shorter names are nicer to deal with in
general.
Latest protobuf.js requires that we pass in the sign value when making
longs from strings, ex: dcodeIO.Long.fromString(id, true);
However, it does the string->long conversion automatically if its given
a string for a fixed64 field, so we can pass our string ids right in!
ftw
Moved all test code into /test. Renamed test.js to crypto_test.js.
(Let's try to keep test files topical.) Merged test_views.html and
test.html into a single test/index.html.
Todo: use Grunt to generate test/index.html from index.html and files
found in /test. Also, write more tests.
We now correctly and opportunistically use the webcrypto API if
available, polyfilling if it's not detected. This change also includes a
layer of abstraction over the webcrypto interface so we no longer have
to deal with key-imports or algorithm names all over the place. Since we
no longer support AES-CTR, code outside this file can simply call
`textsecure.subtle.<encrypt|decrypt|sign>(key, data [, iv])`.
TypedArray.prototype.set doesn't handle ArrayBuffers correctly (it
writes all zeros). Instead, wrap each ArrayBuffer in a typed array
for concatenation.
processData (default: true)
Type: Boolean
By default, data passed in to the data option as an object (technically,
anything other than a string) will be processed and transformed into a
query string, fitting to the default content-type
"application/x-www-form-urlencoded". If you want to send a DOMDocument,
or other non-processed data, set this option to false.
https://api.jquery.com/jQuery.ajax/
Parse attachment ids out of the attachment pointer url and return them
as strings because the copy parsed by JSON suffers a loss of precision.
Convert them to and from the format expected by the protobuf using
facilities from decodeIO.Long.
Sadly, we are not quite compliant with the WC3 webcrypto spec
due to our insistance on passing around key data in plain old
ArrayBuffers.
Also converted whitespace.
DRY up protobuf declarations and move to a slightly briefer naming
convention.
Also dropped some ArrayBuffer -> string conversions as
ProtoBuf.js handles ArrayBuffers just fine, and in fact, more
efficiently than strings.
Finally, dropped the btoa() wrappers, because that incurs an extra
string -> string conversion before the protobuf's internal string ->
array buffer conversion. In lieu of btoa, we can simply pass in the
optional string encoding argument to the protobuf's decode method,
which in these cases should be 'binary'.
Related: #17
All the group messages were being sent to the last recipient in the
list, due to the persistence of `var number` in later loops and async
calls. An easy mistake to make, when you use for instead of each.
An exception is thrown when protobuf tries to encode a number as the
group id, which is declared to have type 'bytes'.
Fix by make it an ArrayBuffer instead, and increase the length to 16,
which is what the Android client uses:
c632b32ff8/src/org/thoughtcrime/securesms/database/GroupDatabase.java (L222)
Previously, if calling createNewGroup with an undefined groupId,
no groupId was generated.
This occurred because no entry for "group" + undefined exists in
localStorage, which caused this code to think undefined was a
valid group id.
Fixed by adding `|| groupId == undefined` to the while clause.
Also decoupled the groupId collision check for clarity.
When codes are sent they are formatted as xxx-xxx. Previously when I
would paste these from GVoice they failed validation thanks to the dash
and whatever whitespace I happened to grab.
Also,
* moved fetch out of the list view
* removed unused #last() function
* put test setup lines in their own tiny file.
* added data-cover to view script tags for code coveage reports.
The layout class is the only class that should have knowledge of
page-level constant markup, such as #gutter and #contacts, and
should be pretty much the only place we find elements by id (with
the exception of template elements).
This change removes references to #gutter from views. Rather than
hardcoding assumptions about page layout, view elements should
ask the layout to insert themselves into the main content area by
calling Whisper.Layout.setContent.
Each conversation views now manages its own separate elements
rather than all binding to a shared #conversation element, and
similarly for message composition ui.
Also includes the beginnings of group creation UI (not working yet),
featuring bootstrap-tagsinput field for entering group recipients
Let ConversationListItemView save a reference to its corresponding
ConversationView. This lets it render or delegate/undelegate events
when opening and closing a conversation.
Similarly for ConversationView itself, which contains a MessageListView.
When a thread is 'destroyed' from the UI we delete its messages and mark
the thread as inactive, (in other words, keep it around as contact info).
Additionally, we only load active threads when initializing the UI, and
reactivate threads when new messages are added to them.
Conflicts:
js/models/messages.js
js/models/threads.js
js/views/conversations/show.js
ERHMAGERRRD testing frameworks are so the best. Removed all our custom
code for ensuring test exclusivity and doneness and isolating callbacks
and everything. mocha does it all for us, and makes it pretty.
Also rather than return a long chain of promises that eventually resolve
to truthiness, we now use chai to make assertions about what is good and
right in the world.
Recommended reading:
https://visionmedia.github.io/mochahttp://chaijs.com/api/assert/
There were a few problems.
1. The message event was being triggered in background, not popup
2. The initial message/thread fetches from localStorage were mis-ordered
3. The timestamp wasn't being extracted from the right place
4. #3 caused messages to fail validation and not be saved
1-3 are fixed. To address 4 I switched validate() to log a warning
instead of preventing save.
Adds thread model/collection for managing conversation-level state, such
as unreadCounts, group membership, thread order, etc... plus various UI
improvements enabled by thread model, including an improved compose
flow, and thread-destroy button.
Adds Whisper.notify for presenting messages to the user in an orderly
fashion. Currently using a growl-style fade in/out effect.
Also some housekeeping:
Cut up views into separate files.
Partial fix for formatTimestamp.
Tweaked buttons and other styles.
Slight revert from said commit. We really do need the
IncomingPushMessageSignal protobuf at the UI layer, mostly because
it contains the 'source' attribute, without which we don't know
who sent the message.
Also fix a crash when there are no attachments on a message.
Better load the functions defined in chromium.js before trying to use
them. Hmm.. also, options.js should probably wait for the DOM to load
before it tries to initialize things in it.
The 'sender' field actually holds the recipient for outgoing
messages. Rename that field to 'person', indicating the 2nd
party generically.
Also decouples the thread name from thread recipients at the
view layer, in preparation for group support.
Adds Backbone-based Whisper.Messages model/collection with local storage
extension. Saves sent and received messages in Whisper.Messages instead
of message map. This will assign a unique id to the message and save it
to localStorage.
Adds Backbone-based view to popup.html
Automatically updates itself when new messages are saved to
Whisper.Messages db from the background page.
Added some shiny new styles, and started splitting up css into multiple
files for sanity's sake.
btoa expects a string argument, so when passing it the ArrayBuffer
object returned by getRandomBytes(), it's converted to a string by
calling .toString() on it. This always results in "[object ArrayBuffer]",
effectively resulting in a completely non-random password.
var i was being bound in the closure of this click handler, then
incremented by the for loop, such that its value was 1 by the time the
handler was called, so we were grabbing the message body from, e.g.
$("#input1") when we wanted $("#input0").
* key API changes moxie made because he disliked the other API
* remove atmosphere
* Fix some bugs in the send path, update for new send API
* Send HTML
When included after api.js, fake_api.js inits a FakeWhisperAPI.
FakeWhisperAPI inherits the methods of API, overrides a few, and
then usurps its place as the one true API.
Single device mode successfully "registers" against FakeAPI. Sadly,
multidevice mode has a recursive loop somewhere that makes the callstack
asplode.
The details of the server API are now mostly relegated to api.js, and
accessed through the API container object, improving modularity and
readability, and setting us up to derive a FakeAPI for serverless
development.
