WordPress.org

Make WordPress Core

Opened 2 months ago

Last modified 6 days ago

#43797 assigned enhancement

Logging for GDPR privacy/security

Reported by: xkon Owned by: xkon
Milestone: Awaiting Review Priority: normal
Severity: normal Version:
Component: Privacy Keywords: gdpr 2nd-opinion has-patch privacy-roadmap
Focuses: Cc:

Description

After a little chat in #gdpr-compliance as some questions where raised https://ico.org.uk/for-organisations/guide-to-the-general-data-protection-regulation-gdpr/lawful-basis-for-processing/consent/ was presented.

It seems that we have to create a mechanism so plugins/themes can tap into and record any consents so the admins can keep a proper log of them for later use if needed.

I think it would be prefered if it's somewhere centralized again instead of each plugin keeping it's own log.

Attachments (1)

43797.diff (11.5 KB) - added by xkon 2 months ago.
introducing consent log tool

Download all attachments as: .zip

Change History (30)

This ticket was mentioned in Slack in #gdpr-compliance by xkon. View the logs.


2 months ago

#2 @Shelob9
2 months ago

I'm the author of Caldera Forms, a fairly popular plugin on WordPress.org for making forms. [We are working on adding a field to consent to personally identifying](https://github.com/CalderaWP/Caldera-Forms/issues/2428). It would be very helpful to me, if I had an API like this:

<?php
//$consent_api contains instance of class that serves as API for this info

//Check if we have consent, by email address
$has_consent = $consent_api->has_given_consent( 'hi@hiroy.club' );

//Record consent by email address
$consent_api->add_consent( 'hi@hiroy.club' );

//Remove consent by email address
$consent_api->remove_consent( 'hi@hiroy.club' );

This ticket was mentioned in Slack in #gdpr-compliance by shelob9. View the logs.


2 months ago

#4 @mnelson4
2 months ago

The “WP GDPR framework” already has an implementation of this that sounds pretty good: https://codelight.eu/wordpress-gdpr-framework/developer-docs/ (scroll to “consent”). They mention functions

<?php
gdpr('consent')->register(...)

(so you can register meta information about the consent, like a pretty name, so that data doesn’t need to be repeated),

<?php
$dataSubject = gdpr('data-subject')->getByEmail($_POST['email']);
  $dataSubject->giveConsent('my_custom_consent_slug'); 

(because not all data subjects will be users, and grants specific consent), and a filter for handling what to do when consent is revoked. I haven’t used it myself, but thought I’d mention the prior art.

#5 @xkon
2 months ago

I've been doing something similar with a website that has custom forms / cookies etc, so I thought of converting the code a little bit to see if it would help here at all. I went along the lines of how the requests CPTs where created for the privacy tools but I'm not sure if there's a better/clever way to do it for core. I have a custom admin page that lists all of the consents so the admin can easily who has done what also.

Ofc this might be pretty basic but it has served the websites needs pretty well until now.

I can add this on the user.php that we currently have all of our extra classes if this is ok for a start so we can check / enhance it better.

<?php
class WP_Privacy_Consent_Logs {

        public function consent_exists( $attr ) {

                $args = array(
                        'post_type'  => 'consent_log',
                        'meta_query' => array(
                                'relation'     => 'AND',
                                '_user_email' => array(
                                        'key'   => '_user_email',
                                        'value' => $attr['email_address'],
                                ),
                                '_consent_identifier' => array(
                                        'key'   => '_consent_identifier',
                                        'value' => $attr['identifier'],
                                ),
                        ),
                );

                $query = new WP_Query( $args );

                if ( $query->have_posts() ) {
                        return $query->post->ID;
                } else {
                        return false;
                }

        }

        public function has_consent( $attr ) {

                $exists = $this->consent_exists( array(
                        'email_address' => $attr['email_address'],
                        'identifier'    => $attr['identifier'],
                ));

                if ( $exists ) {
                        if ( 'yes' === get_post_meta( $exists ,'_consent_status', true ) ) {
                                return true;
                        }
                } else {
                        return false;
                }
        }

        public function add_consent( $attr ) {

                $exists = $this->consent_exists( array(
                        'email_address' => $attr['email_address'],
                        'identifier'    => $attr['identifier'],
                ));

                if ( ! $exists ) {

                        $user_id = 0;

                        $consent = wp_insert_post( array(
                                'post_author'   => $user_id,
                                'post_status'   => 'publish',
                                'post_type'     => 'consent_log',
                                'post_date'     => current_time( 'mysql', false ),
                                'post_date_gmt' => current_time( 'mysql', true ),
                        ), true );

                        update_post_meta( $consent, '_user_email', $attr['email_address'] );
                        update_post_meta( $consent, '_consent_identifier', $attr['identifier'] );
                        update_post_meta( $consent, '_consent_status', $attr['accepted'] );

                        return true;

                } else {
                        return false;
                }

        }

        public function remove_consent( $attr ) {

                $exists = $this->consent_exists( array(
                        'email_address' => $attr['email_address'],
                        'identifier'    => $attr['identifier'],
                ));

                if ( $exists ) {
                        wp_delete_post( $exists );
                        return true;
                } else {
                        return false;
                }

        }

        public function update_consent( $attr ) {

                if ( ! empty( $attr['accepted'] ) ) {
                        $exists = $this->consent_exists( array(
                                'email_address' => $attr['email_address'],
                                'identifier'    => $attr['identifier'],
                        ));

                        if ( $exists ) {
                                update_post_meta( $exists, '_consent_status', $attr['accepted'] );
                                return true;
                        } else {
                                return false;
                        }
                }

        }

}

I'm using it this way pretty much:

<?php
// consent_exists 
$consent = new WP_Privacy_Consent_Logs();
$args = array(
        'email_address' => 'test@test.test',
        'identifier'    => 'cookie_form_1',
);

$check = $consent->consent_exists( $args );

if ( $check ) {
        error_log( 'exists' );
} else {
        error_log( 'exists not' );
}

// has_consent 
$consent = new WP_Privacy_Consent_Logs();
$args = array(
        'email_address' => 'test@test.test',
        'identifier'    => 'cookie_form_1',
);

$check = $consent->has_consent( $args );

if ( $check ) {
        error_log( 'has' );
} else {
        error_log( 'has not' );
}


// add_consent
$consent = new WP_Privacy_Consent_Logs();
$args = array(
        'email_address' => 'test@test.test',
        'identifier'    => 'cookie_form_1',
        'accepted'      => 'no',
);

$check = $consent->add_consent( $args );

if ( $check ) {
        error_log( 'added' );
} else {
        error_log( 'not added' );
}


// remove_consent
$consent = new WP_Privacy_Consent_Logs();
$args = array(
        'email_address' => 'test@test.test',
        'identifier'    => 'cookie_form_1',
);

$check = $consent->remove_consent( $args );

if ( $check ) {
        error_log( 'removed' );
} else {
        error_log( 'not removed' );
}

// update_consent
$consent = new WP_Privacy_Consent_Logs();
$args = array(
        'email_address' => 'test@test.test',
        'identifier'    => 'cookie_form_1',
        'accepted'      => 'yes',
);

$check = $consent->update_consent( $args );

if ( $check ) {
        error_log( 'updated' );
} else {
        error_log( 'not updated' );
}
Last edited 2 months ago by xkon (previous) (diff)

This ticket was mentioned in Slack in #gdpr-compliance by xkon. View the logs.


2 months ago

#7 @mnelson4
2 months ago

Yeah @xkon I think that satisfies the requirement. Personally I'd be inclined to have two arguments for the functions, the first being an email and the second being the consent's identifier. That would be clearer than providing an array IMO.

Using the posts table seems a bit heavy-handed because this data isn't very post-y (most columns are unused, and all the important data needs to be stored separately), but I don't have any better suggestions than using a custom table. Usermeta won't work because we need to track consent for non-users (eg commenters). Storing this info as a option seems barbaric.

TBH I'd be most happy with a custom table but I realize that has a lot of migration work etc.

#8 @xkon
2 months ago

Thanks for your input @mnelson4 I've actually added a small repo and converted this into a plugin so it's easier for anyone that wanted to have a look to see how this feels or expand it for the ease of use ( https://github.com/mrxkon/consent-log ).

As for the custom table, I left it as is with a CPT since we used CPTs for the exports other tools also for now.

An extra note after a little bit of discussion the 'feeling' about this is that it might get abused by plugins overchecking consents ( although each plugin should add a unique ID per consent so there shouldn't be a problem there as any other plugin wouldn't know where to look ) + the idea of keeping 'extra' data such as e-mails etc.

From one hand you allow the user to get anonymized and from the other you have to keep his consent log to prove that at some past point he had accepted a consent, if the consent is anonymized as well how are you going to prove it in the first place? It seems that some things here are going against each other if I have it down correctly.

If we find a way to rule out the possibility of doing something actually 'against' the regulation so we can be safe and this is something that more people actually need we can surely check it out for the future.

---

Since my position is also conflicting ( by giving code and saying to wait a bit longer ), I'll say again that I had made something similar to this recently for a client as it was requested by his lawyers. It doesn't mean that even they are 100% correct though :) .

#9 @desrosj
2 months ago

  • Owner set to xkon
  • Status changed from new to assigned

#10 @mnelson4
2 months ago

As for the custom table, I left it as is with a CPT since we used CPTs for the exports other tools also for now.

Ya I see this follows that precedent, and that's probably fine.

And I'll try to take the plugin for a spin.

@xkon
2 months ago

introducing consent log tool

#11 follow-up: @xkon
2 months ago

  • Keywords 2nd-opinion has-patch added

After back & forth discussions about this, the opinions are somewhat in the middle of if this should be in core or not.

I'm in favor of trying to provide as much as possible built-in so we know that the Admins have a single page to view for their proof of consents, even if core itself doesn't actually need it on it's own.

In 43797.diff I've made a first pass of moving the plugin into core the same way as the other privacy tools are handled at the moment.

  • This creates a new consent_log CPT that can be used pretty much to log anything ( not only consents to be honest ).
  • It adds 2 new post statuses consent_accepted - consent_declined
  • It adds a Consent Log page under Tools with the option to remove all consents for a certain User by the admin.

There are 4 functions requires 3 things:

User ID: you can place here whatever, either email, id, or any other identifier that fits your needs.

Consent ID: This should be unique as a prevention of plugins overriding each other and taping into / reading different consents to avoid making extra ones. This would be considered misuse. The unique consent ID will be provided on any given form as a random maybe and it will be the plugins/themes/authors responsibility to keep track of so it can re-check it.

Consent Status: either consent_accepted, or consent_declined

<?php
$the_consent = new Consent_Log();

$consent = $the_consent->has_consent( 'test@test.oo', 'unique_form_id_1' );

$consent = $the_consent->create_consent( 'test@test.oo', 'unique_form_id_1', 'consent_accepted' );

$consent = $the_consent->update_consent( 'test@test.oo', 'unique_form_id_1', 'consent_declined' );

$consent = $the_consent->remove_consent( 'test@test.oo', 'unique_form_id_1' );

I'm sure there are more stuff to add / fix / alter but I think this is a decent start for this kind of Tool.

This ticket was mentioned in Slack in #core by allendav. View the logs.


8 weeks ago

#13 in reply to: ↑ 11 ; follow-up: @ericdaams
8 weeks ago

Storing consents in the posts table seems fraught with peril to me, if I understand the purpose behind consents correctly.

For example, if I have a cookie notice that is displayed to all visitors, and this is agreed to by the visitor, would this be something that I could/should store a consent for?

If that's the case, then every single visitor to my website who accepts this will end up having a consent entry created for them; that's a whole lot of bulk added to the posts table.

I understand that there is precedent for doing things as CPTs, say with requests for data erasure or export, but those requests are likely to be far fewer; consents is something that can very likely be hit again and again.

On the topic of developing this in core vs. as a plugin, would this be an appropriate candidate for a feature plugin? That might allow us to develop it in a less rushed fashion, with a custom table designed just for it.

#14 in reply to: ↑ 13 @xkon
8 weeks ago

Replying to ericdaams (also adding general things discussed in slack so we can have them here as well):

Storing consents in the posts table seems fraught with peril to me, if I understand the purpose behind consents correctly. I understand that there is precedent for doing things as CPTs, say with requests for data erasure or export, but those requests are likely to be far fewer; consents is something that can very likely be hit again and again.

This first patch was made from a plugin that I already made / using for my clients as mentioned above as it was using CPTs already. One of the websites already has over 3k consents within a matter of days logged and they're moving up so I can easily see the concerns on this.

I pretty much took some time to match it with our current way of providing tools so you could just hit apply patch and check how things work ( at least for me :) ) the last weeks after I saw that @Shelob9 and @mnelson4 mentioned pretty much the same stuff that I was already using.

For example, if I have a cookie notice that is displayed to all visitors, and this is agreed to by the visitor, would this be something that I could/should store a consent for?

Depends on if you want proof for it or yes you might as well log everything.

If that's the case, then every single visitor to my website who accepts this will end up having a consent entry created for them; that's a whole lot of bulk added to the posts table.

It is but you can avoid logging if you want. It's not 'automated' and won't be (by core) in any way for the consent part at least. We provide the means to do it, but it's up to any Admin to use them or not.

On the topic of developing this in core vs. as a plugin, would this be an appropriate candidate for a feature plugin? That might allow us to develop it in a less rushed fashion, with a custom table designed just for it.

So there have been some chats in slack already and the thinking around this is to make it even more global to log more things and not only Consents. That being said the records will be even more and the logging will be 'endless' but that's how logs work + that's up to any admin to decide for how long the records will be kept for.

This is in general something that I will continue to try and have it in Core as it's apparently needed from the regulation + a privacy/security measure that the Core itself can easily provide with an api that everybody can use and follow.

Examples that have been mentioned:

  • Consents
  • User Logs (Registration/Profile Updates/Deletion)
  • Privacy Tools Logs ( to port them here for centralization )
  • Page/Post Editing ( User/Date -> pretty much like the revision table is viewed inside an Edit page at the moment, but again this will just be a mention + centralized for a quick review )

As you can imagine the list might even get bigger/smaller etc it's still on early discussions.

#15 @xkon
7 weeks ago

  • Summary changed from Consent Logging to Logging for GDPR privacy/security

This ticket was mentioned in Slack in #gdpr-compliance by eric.daams. View the logs.


7 weeks ago

This ticket was mentioned in Slack in #gdpr-compliance by allendav. View the logs.


7 weeks ago

This ticket was mentioned in Slack in #gdpr-compliance by brento. View the logs.


7 weeks ago

This ticket was mentioned in Slack in #gdpr-compliance by allendav. View the logs.


6 weeks ago

This ticket was mentioned in Slack in #gdpr-compliance by allendav. View the logs.


6 weeks ago

This ticket was mentioned in Slack in #gdpr-compliance by desrosj. View the logs.


6 weeks ago

#22 @azaozz
5 weeks ago

IMHO the first step here is to not let admins (or anybody else) remove/delete any of the privacy "requests", just like post revisions are not removable. They are the current log, both completed and failed.

If somebody don't want to use them in their logging/audit trail capacity, they can add a plugin that will let them delete these requests / CPT posts. Same case like post revisions :)

Last edited 5 weeks ago by azaozz (previous) (diff)

#23 @desrosj
5 weeks ago

  • Component changed from General to Privacy

Moving to the new Privacy component.

#24 @stiofansisland
4 weeks ago

I think this really needs to be in core.

I think it needs it's own table so not to clutter the posts table.

Whats the chances of this being in core anytime soon? If not can we start working on a featured plugin for this? (i would like to help)

#25 @xkon
4 weeks ago

@azaozz yeah, none of the log entries should be deletable (or at least not 'easily' deletable). Also another thinking of a more concrete 'logging' would be to not update a record but just add a new one.

So each 'action' ( either consent, export, erasure etc ) is kept into it's own record that would give you a better timeline of what happened as well.

Now that we have the release live I'll be adding some more things as well as they're needed around here (in the office) and also change some minor things on the previous patch just for the sake of previewing.

I'll try to get an update for a more complete 'log-type' of list asap to see what we're missing and how to get everything going.

This ticket was mentioned in Slack in #gdpr-compliance by stiofansisland. View the logs.


4 weeks ago

This ticket was mentioned in Slack in #gdpr-compliance by desrosj. View the logs.


3 weeks ago

#28 @desrosj
3 weeks ago

Related: #44043.

#29 @desrosj
6 days ago

  • Keywords privacy-roadmap added
Note: See TracTickets for help on using tickets.